2024-09-10T23:59:42.291 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T23:59:42.319 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T23:59:42.425 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899823' kernel: client: branch: testing flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyZ/J+wKwUqbCRD/fx5aA/lgyOBL0UQ6l3/ildjpM7Akj+r1zRWGcwBieyBefDY/phZaBt6M3gxIj3XVIW9tVo= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJcMiy7CpGc1SUXvatBLq+P6sflZLbJikqR4SG6zOkelBHD9+2ikh2Bpwkyqm6HysDMCZwMmlpPESKT0pgKHhqg= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF78jWH8I1Tlbsd8ouERwdzE1lMh6v8QayC4LbPK5sJjBUUCNZoNEZ2FdHAIF9ta1GaORxftBFG35hS9mT1xf5c= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T23:59:42.425 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T23:59:42.426 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T23:59:42.426 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T23:59:42.427 INFO:teuthology.task.internal:Checking packages... 2024-09-10T23:59:42.448 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T23:59:42.449 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T23:59:42.449 INFO:teuthology.packaging:ref: None 2024-09-10T23:59:42.449 INFO:teuthology.packaging:tag: None 2024-09-10T23:59:42.449 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T23:59:42.449 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T23:59:42.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T23:59:42.603 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T23:59:42.605 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T23:59:42.610 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T23:59:42.610 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T23:59:42.616 INFO:teuthology.task.internal:Saving configuration 2024-09-10T23:59:42.635 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T23:59:42.641 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T23:59:42.674 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:54:18.788015', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKyZ/J+wKwUqbCRD/fx5aA/lgyOBL0UQ6l3/ildjpM7Akj+r1zRWGcwBieyBefDY/phZaBt6M3gxIj3XVIW9tVo='} 2024-09-10T23:59:42.706 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:54:18.789816', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJcMiy7CpGc1SUXvatBLq+P6sflZLbJikqR4SG6zOkelBHD9+2ikh2Bpwkyqm6HysDMCZwMmlpPESKT0pgKHhqg='} 2024-09-10T23:59:42.740 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 23:54:18.785957', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF78jWH8I1Tlbsd8ouERwdzE1lMh6v8QayC4LbPK5sJjBUUCNZoNEZ2FdHAIF9ta1GaORxftBFG35hS9mT1xf5c='} 2024-09-10T23:59:42.740 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T23:59:42.801 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T23:59:42.801 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T23:59:42.801 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T23:59:42.801 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T23:59:43.042 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe961053be0>, signals=[15]) 2024-09-10T23:59:43.042 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T23:59:43.071 INFO:teuthology.task.internal:Opening connections... 2024-09-10T23:59:43.071 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-09-10T23:59:43.072 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:59:43.160 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-09-10T23:59:43.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:59:43.246 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-10T23:59:43.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T23:59:43.344 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T23:59:43.351 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-09-10T23:59:43.370 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-09-10T23:59:43.370 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T23:59:43.428 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-09-10T23:59:43.429 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-09-10T23:59:43.429 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T23:59:43.429 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-09-10T23:59:43.429 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T23:59:43.429 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T23:59:43.429 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T23:59:43.429 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-09-10T23:59:43.457 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-09-10T23:59:43.476 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-09-10T23:59:43.476 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-09-10T23:59:43.534 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-09-10T23:59:43.535 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T23:59:43.536 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T23:59:43.536 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T23:59:43.536 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-09-10T23:59:43.565 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-10T23:59:43.584 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-10T23:59:43.584 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-10T23:59:43.642 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-09-10T23:59:43.642 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-09-10T23:59:43.642 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T23:59:43.643 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T23:59:43.644 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-10T23:59:43.674 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T23:59:43.684 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T23:59:43.690 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T23:59:43.690 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:59:43.693 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:59:43.695 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-10T23:59:43.712 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T23:59:43.718 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T23:59:43.719 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:59:43.750 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:59:43.753 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-10T23:59:43.836 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T23:59:43.938 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T23:59:43.952 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T23:59:43.953 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-10T23:59:43.953 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-10T23:59:43.953 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T23:59:43.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T23:59:43.971 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T23:59:44.161 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:59:44.161 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T23:59:44.161 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-10T23:59:44.179 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-503.el9.x86_64 2024-09-10T23:59:44.180 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T23:59:44.180 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T23:59:44.181 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum upgrade -y linux-firmware 2024-09-10T23:59:48.674 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.2 MB 00:02 2024-09-11T00:00:09.629 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 20 MB 00:19 2024-09-11T00:00:14.596 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.5 MB 00:01 2024-09-11T00:00:16.105 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-11T00:00:17.426 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-11T00:00:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T00:00:24.931 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-11T00:00:24.932 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-09-11T00:00:24.932 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-11T00:00:25.044 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-11T00:00:25.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-11T00:00:25.204 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-11T00:00:25.204 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-11T00:00:25.207 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi103.front.sepia.ceph.com to complete... 2024-09-11T00:00:31.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11 00:00:31 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-11T00:00:31.505 INFO:teuthology.orchestra.run.smithi103.stderr:FINISHED --2024-09-11 00:00:31-- 2024-09-11T00:00:31.505 INFO:teuthology.orchestra.run.smithi103.stderr:Total wall clock time: 6.3s 2024-09-11T00:00:31.505 INFO:teuthology.orchestra.run.smithi103.stderr:Downloaded: 1 files, 343M in 6.0s (57.2 MB/s) 2024-09-11T00:00:31.506 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-11T00:00:31.507 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-11T00:00:33.616 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying... ######################################## 2024-09-11T00:00:33.836 INFO:teuthology.orchestra.run.smithi103.stdout:Preparing... ######################################## 2024-09-11T00:00:33.949 INFO:teuthology.orchestra.run.smithi103.stdout:Updating / installing... 2024-09-11T00:00:41.969 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-11T00:01:22.693 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-11T00:01:22.693 DEBUG:teuthology.orchestra.run.smithi103:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-11T00:01:22.796 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-11T00:01:22.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-11T00:01:22.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-11T00:01:22.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-11T00:01:22.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-11T00:01:22.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-11T00:01:22.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-11T00:01:22.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-11T00:01:22.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-11T00:01:22.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-11T00:01:22.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-11T00:01:22.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-11T00:01:22.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-11T00:01:22.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-11T00:01:22.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-11T00:01:22.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-11T00:01:22.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-11T00:01:22.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-11T00:01:22.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-11T00:01:22.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-11T00:01:22.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-11T00:01:22.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-11T00:01:22.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-11T00:01:22.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-11T00:01:22.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-11T00:01:22.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-11T00:01:22.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-11T00:01:22.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-11T00:01:22.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-11T00:01:22.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-11T00:01:22.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-11T00:01:22.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-11T00:01:22.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-11T00:01:22.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-11T00:01:22.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-11T00:01:22.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-11T00:01:22.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-11T00:01:22.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-11T00:01:22.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-11T00:01:22.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-11T00:01:22.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-11T00:01:22.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-11T00:01:22.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-11T00:01:22.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-11T00:01:22.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-11T00:01:22.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-11T00:01:22.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-11T00:01:22.840 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-11T00:01:22.841 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-11T00:01:22.842 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-11T00:01:22.843 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-11T00:01:22.844 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-11T00:01:22.845 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-11T00:01:22.845 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-11T00:01:22.845 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-11T00:01:22.845 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-11T00:01:22.846 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-11T00:01:22.847 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-11T00:01:22.848 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-11T00:01:22.849 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-11T00:01:22.850 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-11T00:01:22.851 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-11T00:01:22.852 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-11T00:01:22.853 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-11T00:01:22.854 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-11T00:01:22.855 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-11T00:01:22.856 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-11T00:01:22.856 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-11T00:01:22.856 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-11T00:01:22.856 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-11T00:01:22.856 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-11T00:01:22.856 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-11T00:01:22.856 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-11T00:01:22.857 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-11T00:01:22.858 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-11T00:01:22.859 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-11T00:01:22.860 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-11T00:01:22.861 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-11T00:01:22.862 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-11T00:01:22.863 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-11T00:01:22.864 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-11T00:01:22.865 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-11T00:01:22.865 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-11T00:01:22.865 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-11T00:01:22.865 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-11T00:01:22.865 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-11T00:01:22.865 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-11T00:01:22.865 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-11T00:01:22.866 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-11T00:01:22.867 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-11T00:01:22.867 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-11T00:01:22.867 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-11T00:01:22.867 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-11T00:01:22.867 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-11T00:01:22.867 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-11T00:01:22.867 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-11T00:01:22.868 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-11T00:01:22.869 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-11T00:01:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-11T00:01:22.871 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-11T00:01:22.871 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-11T00:01:22.871 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-11T00:01:22.871 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-11T00:01:22.871 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-11T00:01:22.872 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-11T00:01:22.873 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-11T00:01:22.874 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-11T00:01:22.875 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-11T00:01:22.876 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-11T00:01:22.877 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-11T00:01:22.878 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-11T00:01:22.879 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-11T00:01:22.880 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-11T00:01:22.881 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-11T00:01:22.882 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-11T00:01:22.883 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-11T00:01:22.884 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-11T00:01:22.885 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-11T00:01:22.886 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-11T00:01:22.887 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-11T00:01:22.888 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-11T00:01:22.889 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-11T00:01:22.890 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-11T00:01:22.891 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-11T00:01:22.892 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-11T00:01:22.893 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-11T00:01:22.894 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-11T00:01:22.895 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-11T00:01:22.896 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-11T00:01:22.897 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-11T00:01:22.898 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-11T00:01:22.899 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-11T00:01:22.900 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-11T00:01:22.901 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-11T00:01:22.902 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-11T00:01:22.903 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-11T00:01:22.904 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-11T00:01:22.905 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-11T00:01:22.906 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-11T00:01:22.907 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-11T00:01:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-11T00:01:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-11T00:01:22.910 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-11T00:01:22.911 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-11T00:01:22.912 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-11T00:01:22.913 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-11T00:01:22.914 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-11T00:01:22.915 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-11T00:01:22.916 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-11T00:01:22.917 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-11T00:01:22.918 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-11T00:01:22.919 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-11T00:01:22.920 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-11T00:01:22.921 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-11T00:01:22.922 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-11T00:01:22.923 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-11T00:01:22.924 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-11T00:01:22.925 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-11T00:01:22.926 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-11T00:01:22.927 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-11T00:01:22.928 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-11T00:01:22.929 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-11T00:01:22.930 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-11T00:01:22.931 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-11T00:01:22.932 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-11T00:01:22.933 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-11T00:01:22.934 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-11T00:01:22.935 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-11T00:01:22.936 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-11T00:01:22.937 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-11T00:01:22.938 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-11T00:01:22.939 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-11T00:01:22.940 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-11T00:01:22.941 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-11T00:01:22.942 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-11T00:01:22.943 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-11T00:01:22.944 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-11T00:01:22.945 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-11T00:01:22.946 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-11T00:01:22.947 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-11T00:01:22.948 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-11T00:01:22.949 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-11T00:01:22.950 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-11T00:01:22.951 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-11T00:01:22.952 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-11T00:01:22.953 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-11T00:01:22.954 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-11T00:01:22.955 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-11T00:01:22.956 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-11T00:01:22.957 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-11T00:01:22.958 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-11T00:01:22.959 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-11T00:01:22.960 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-11T00:01:22.961 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-11T00:01:22.962 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-11T00:01:22.963 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-11T00:01:22.964 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-11T00:01:22.965 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-11T00:01:22.966 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-11T00:01:22.967 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-11T00:01:22.968 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-11T00:01:22.969 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-11T00:01:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-11T00:01:22.971 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-11T00:01:22.972 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-11T00:01:22.977 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:22.977 DEBUG:teuthology.orchestra.run.smithi103:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-11T00:01:23.002 INFO:teuthology.orchestra.run.smithi103.stdout:postinstall scriptlet (using /bin/sh): 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout:for file in vmlinuz System.map config; do 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout: fi 2024-09-11T00:01:23.003 INFO:teuthology.orchestra.run.smithi103.stdout:done 2024-09-11T00:01:23.004 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-11T00:01:23.004 INFO:teuthology.orchestra.run.smithi103.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-11T00:01:23.004 INFO:teuthology.orchestra.run.smithi103.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-11T00:01:23.004 INFO:teuthology.orchestra.run.smithi103.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-11T00:01:23.004 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-11T00:01:23.004 INFO:teuthology.orchestra.run.smithi103.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-11T00:01:23.004 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:23.005 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-11T00:01:23.005 INFO:teuthology.orchestra.run.smithi103.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-11T00:01:23.005 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-11T00:01:23.005 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:23.005 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-09-11T00:01:23.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 82.el9 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8274670 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T00:01:23.093 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:01:23.094 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T00:01:23.095 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T00:01:23.095 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:23.095 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T00:01:23.665 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-09-11T00:01:24.988 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T00:01:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-09-11T00:01:25.018 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-09-11T00:01:25.056 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.27qABzP4T7 2024-09-11T00:01:25.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.27qABzP4T7 2024-09-11T00:01:25.123 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.27qABzP4T7 2024-09-11T00:01:25.262 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.27qABzP4T7 is 6KB 2024-09-11T00:01:25.292 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.27qABzP4T7 2024-09-11T00:01:25.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T00:01:25.369 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T00:01:25.369 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T00:01:25.369 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T00:01:25.369 INFO:teuthology.orchestra.run.smithi103.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-11T00:01:25.371 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:01:25.568 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-09-11T00:01:55.571 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T00:01:55.572 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-11T00:01:55.573 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:02:13.953 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-11T00:02:22.954 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-11T00:02:22.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:02:26.049 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-09-11T00:02:38.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-11T00:02:38.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:03:38.062 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T00:03:53.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-11T00:03:53.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:03:53.531 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-11T00:03:54.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-11T00:03:54.019 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-11T00:03:54.019 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-09-11T00:03:54.053 INFO:teuthology.orchestra.run.smithi103.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:03:54.053 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T00:03:54.053 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T00:03:54.053 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T00:03:55.055 DEBUG:teuthology.parallel:result is None 2024-09-11T00:03:55.055 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T00:03:55.065 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T00:03:55.065 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T00:03:55.069 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T00:03:55.072 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T00:03:55.103 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T00:03:55.118 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T00:03:55.124 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T00:03:55.125 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T00:03:55.147 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T00:03:55.149 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T00:03:55.215 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T00:03:55.224 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T00:03:55.224 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T00:03:55.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:03:55.241 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T00:03:55.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:03:55.259 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T00:03:55.263 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T00:03:55.285 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T00:03:55.320 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:03:55.336 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:03:55.337 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:03:55.352 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:03:55.360 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:03:55.388 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T00:03:55.391 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T00:03:55.446 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T00:03:55.447 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T00:03:55.450 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T00:03:55.452 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T00:03:55.503 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T00:03:55.522 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T00:03:55.522 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T00:03:55.526 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T00:03:55.528 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T00:03:55.556 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:03:55.666 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:03:55.738 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:03:55.808 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:03:55.872 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:03:55.872 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T00:03:55.926 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:03:55.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:03:56.034 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:03:56.086 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:03:56.120 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:03:56.120 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T00:03:56.187 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:03:56.241 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T00:03:56.288 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:03:56.382 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T00:03:56.417 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:03:56.417 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T00:03:56.485 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-09-11T00:03:56.489 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-09-11T00:03:56.492 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-11T00:03:56.527 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:03:56.562 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:03:56.563 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:03:57.074 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T00:03:57.079 INFO:teuthology.task.internal:Starting timer... 2024-09-11T00:03:57.080 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T00:03:57.092 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T00:03:57.099 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T00:03:57.100 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-09-11T00:03:57.180 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-09-11T00:03:57.184 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-09-11T00:03:57.253 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-09-11T00:03:57.255 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-09-11T00:03:57.321 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-09-11T00:03:57.323 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T00:03:57.323 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-09-11T00:03:57.359 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-09-11T00:03:57.359 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-09-11T00:03:57.385 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-09-11T00:03:57.386 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-09-11T00:03:57.411 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-09-11T00:03:57.411 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-09-11T00:03:57.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:03:57.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:03:57.470 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:03:57.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:03:57.499 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:03:57.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:03:57.530 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T00:03:57.530 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-09-11T00:03:57.611 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-09-11T00:03:57.650 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-09-11T00:03:57.683 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T00:03:57.841 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T00:03:57.854 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T00:03:57.855 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi103.front.sepia.ceph.com,smithi117.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T00:08:31.549 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-09-11T00:08:31.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-11T00:08:31.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:08:31.686 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-09-11T00:08:31.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-09-11T00:08:31.800 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-11T00:08:31.801 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:08:31.885 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-11T00:08:31.969 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-11T00:08:31.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-11T00:08:31.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:08:32.062 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-11T00:08:32.148 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-11T00:08:32.149 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T00:08:32.308 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T00:08:32.308 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T00:08:32.309 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:08:32.313 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T00:08:32.313 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:08:32.315 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T00:08:32.316 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:08:32.346 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T00:08:32.346 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T00:08:32.361 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T00:08:32.362 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T00:08:32.371 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T00:08:32.385 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-09-11T00:08:32.386 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-09-11T00:08:32.397 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-09-11T00:08:32.398 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-09-11T00:08:32.402 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T00:08:32.411 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T00:08:32.412 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T00:08:32.425 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T00:08:32.426 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T00:08:32.452 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-09-11T00:08:32.476 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-09-11T00:08:32.480 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:08:32.484 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:08:32.484 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-11T00:08:32.484 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.484 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.484 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.484 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.485 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:08:32.488 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:08:32.488 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-11T00:08:32.488 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.488 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.488 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.488 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.506 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T00:08:32.535 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T00:08:32.635 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:08:32.643 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:08:32.643 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-11T00:08:32.643 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.643 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.643 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.644 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:08:32.644 INFO:teuthology.run_tasks:Running task install... 2024-09-11T00:08:32.667 DEBUG:teuthology.task.install:project ceph 2024-09-11T00:08:32.667 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-11T00:08:32.667 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-11T00:08:32.667 INFO:teuthology.task.install:Using flavor: default 2024-09-11T00:08:32.673 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T00:08:32.673 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T00:08:32.673 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-11T00:08:32.674 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:08:32.676 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-11T00:08:32.676 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:08:32.677 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-11T00:08:32.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:08:32.831 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-11T00:08:32.831 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-11T00:08:32.846 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-11T00:08:32.846 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-11T00:08:32.848 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-11T00:08:32.848 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-11T00:08:32.954 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T00:08:32.955 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:08:32.955 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T00:08:32.968 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T00:08:32.968 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:08:32.968 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T00:08:32.977 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T00:08:32.978 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:08:32.978 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T00:08:32.981 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T00:08:32.981 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T00:08:32.995 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T00:08:32.995 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T00:08:33.034 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T00:08:33.034 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T00:08:33.048 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T00:08:33.064 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T00:08:33.122 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T00:08:33.131 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T00:08:33.140 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T00:08:33.186 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2024-09-11T00:08:33.188 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2024-09-11T00:08:33.204 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-09-11T00:08:33.206 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-09-11T00:08:33.209 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T00:08:33.264 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-09-11T00:08:33.267 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-09-11T00:08:33.412 INFO:teuthology.orchestra.run.smithi117.stdout:45 files removed 2024-09-11T00:08:33.434 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-09-11T00:08:33.439 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T00:08:33.461 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T00:08:33.545 INFO:teuthology.orchestra.run.smithi103.stdout:45 files removed 2024-09-11T00:08:33.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T00:08:33.996 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-11T00:08:33.996 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 326 kB/s | 84 kB 00:00 2024-09-11T00:08:34.288 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-11T00:08:34.288 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-11T00:08:34.310 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 249 kB/s | 84 kB 00:00 2024-09-11T00:08:34.564 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-11T00:08:34.564 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-11T00:08:34.653 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 81 kB/s | 24 kB 00:00 2024-09-11T00:08:34.970 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-09-11T00:08:35.147 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-11T00:08:36.389 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-11T00:08:36.650 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.2 MB 00:01 2024-09-11T00:08:37.482 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-11T00:08:38.455 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-11T00:08:38.607 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-11T00:08:42.074 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-11T00:08:42.899 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-11T00:08:43.173 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.5 MB 00:00 2024-09-11T00:08:43.716 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-11T00:08:44.558 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-11T00:08:44.678 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-11T00:08:44.982 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-11T00:08:45.362 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-11T00:08:45.987 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-11T00:08:50.034 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T00:08:50.834 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2024-09-11T00:08:52.310 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:08:52.311 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:08:52.322 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T00:08:52.322 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T00:08:52.323 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T00:08:52.324 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T00:08:52.326 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T00:08:52.328 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T00:08:52.329 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T00:08:52.385 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-09-11T00:08:52.392 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-11T00:08:52.402 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-11T00:08:52.403 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T00:08:52.404 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-11T00:08:52.405 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T00:08:52.406 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-11T00:08:52.407 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-11T00:08:52.408 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T00:08:52.409 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T00:08:52.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T00:08:52.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T00:08:52.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T00:08:52.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T00:08:52.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T00:08:52.415 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout:Installing weak dependencies: 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout:======================================================================================= 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout:Install 149 Packages 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 8 Packages 2024-09-11T00:08:52.416 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:08:52.417 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 240 M 2024-09-11T00:08:52.417 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-11T00:08:53.123 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:08:53.124 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:08:53.138 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T00:08:53.138 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T00:08:53.139 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T00:08:53.140 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T00:08:53.142 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T00:08:53.145 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T00:08:53.146 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T00:08:53.168 INFO:teuthology.orchestra.run.smithi117.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 54 kB/s | 6.6 kB 00:00 2024-09-11T00:08:53.210 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-11T00:08:53.224 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-11T00:08:53.225 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T00:08:53.226 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-09-11T00:08:53.227 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T00:08:53.228 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T00:08:53.229 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T00:08:53.230 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T00:08:53.231 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T00:08:53.232 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T00:08:53.233 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T00:08:53.234 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T00:08:53.235 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T00:08:53.236 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout:Install 149 Packages 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 8 Packages 2024-09-11T00:08:53.237 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:08:53.238 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 240 M 2024-09-11T00:08:53.238 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-11T00:08:53.394 INFO:teuthology.orchestra.run.smithi117.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 3.9 MB/s | 900 kB 00:00 2024-09-11T00:08:53.469 INFO:teuthology.orchestra.run.smithi117.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.9 MB/s | 145 kB 00:00 2024-09-11T00:08:53.572 INFO:teuthology.orchestra.run.smithi117.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-11T00:08:53.653 INFO:teuthology.orchestra.run.smithi117.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 13 MB/s | 2.4 MB 00:00 2024-09-11T00:08:53.695 INFO:teuthology.orchestra.run.smithi117.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 12 MB/s | 1.5 MB 00:00 2024-09-11T00:08:53.886 INFO:teuthology.orchestra.run.smithi183.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-09-11T00:08:54.095 INFO:teuthology.orchestra.run.smithi117.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 4.8 MB 00:00 2024-09-11T00:08:54.312 INFO:teuthology.orchestra.run.smithi117.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 17 MB/s | 22 MB 00:01 2024-09-11T00:08:54.361 INFO:teuthology.orchestra.run.smithi183.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.9 MB/s | 900 kB 00:00 2024-09-11T00:08:54.362 INFO:teuthology.orchestra.run.smithi117.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 507 kB/s | 25 kB 00:00 2024-09-11T00:08:54.519 INFO:teuthology.orchestra.run.smithi183.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 915 kB/s | 145 kB 00:00 2024-09-11T00:08:54.755 INFO:teuthology.orchestra.run.smithi117.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 17 MB/s | 11 MB 00:00 2024-09-11T00:08:54.921 INFO:teuthology.orchestra.run.smithi117.stdout:(11/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 14 MB/s | 17 MB 00:01 2024-09-11T00:08:54.946 INFO:teuthology.orchestra.run.smithi117.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 1.1 MB/s | 225 kB 00:00 2024-09-11T00:08:54.971 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:08:54.972 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T00:08:54.980 INFO:teuthology.orchestra.run.smithi117.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 462 kB/s | 27 kB 00:00 2024-09-11T00:08:54.984 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T00:08:54.985 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T00:08:54.986 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T00:08:54.987 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T00:08:54.988 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T00:08:54.991 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T00:08:54.992 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T00:08:55.005 INFO:teuthology.orchestra.run.smithi117.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 544 kB/s | 32 kB 00:00 2024-09-11T00:08:55.020 INFO:teuthology.orchestra.run.smithi183.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 4.8 MB/s | 2.4 MB 00:00 2024-09-11T00:08:55.055 INFO:teuthology.orchestra.run.smithi117.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 9.8 MB/s | 755 kB 00:00 2024-09-11T00:08:55.056 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-09-11T00:08:55.081 INFO:teuthology.orchestra.run.smithi117.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:08:55.086 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-11T00:08:55.087 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-11T00:08:55.088 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T00:08:55.089 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-11T00:08:55.090 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T00:08:55.091 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T00:08:55.092 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T00:08:55.093 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T00:08:55.094 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T00:08:55.095 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T00:08:55.096 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T00:08:55.097 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-09-11T00:08:55.098 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-09-11T00:08:55.099 INFO:teuthology.orchestra.run.smithi103.stdout:Install 149 Packages 2024-09-11T00:08:55.099 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 8 Packages 2024-09-11T00:08:55.099 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:08:55.099 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 240 M 2024-09-11T00:08:55.099 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-09-11T00:08:55.123 INFO:teuthology.orchestra.run.smithi117.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 1.9 MB/s | 127 kB 00:00 2024-09-11T00:08:55.156 INFO:teuthology.orchestra.run.smithi117.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 6.6 MB/s | 507 kB 00:00 2024-09-11T00:08:55.223 INFO:teuthology.orchestra.run.smithi117.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 677 kB/s | 45 kB 00:00 2024-09-11T00:08:55.298 INFO:teuthology.orchestra.run.smithi117.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.9 MB/s | 143 kB 00:00 2024-09-11T00:08:55.312 INFO:teuthology.orchestra.run.smithi183.stdout:(5/157): ceph-common-19.3.0-4854.g8f219c47.el9. 13 MB/s | 22 MB 00:01 2024-09-11T00:08:55.370 INFO:teuthology.orchestra.run.smithi183.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-09-11T00:08:55.374 INFO:teuthology.orchestra.run.smithi117.stdout:(21/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 22 MB/s | 5.5 MB 00:00 2024-09-11T00:08:55.399 INFO:teuthology.orchestra.run.smithi117.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.6 MB/s | 161 kB 00:00 2024-09-11T00:08:55.441 INFO:teuthology.orchestra.run.smithi117.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 4.7 MB/s | 321 kB 00:00 2024-09-11T00:08:55.475 INFO:teuthology.orchestra.run.smithi117.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 4.0 MB/s | 306 kB 00:00 2024-09-11T00:08:55.517 INFO:teuthology.orchestra.run.smithi117.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.3 MB/s | 99 kB 00:00 2024-09-11T00:08:55.542 INFO:teuthology.orchestra.run.smithi117.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.2 MB/s | 85 kB 00:00 2024-09-11T00:08:55.579 INFO:teuthology.orchestra.run.smithi183.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 18 MB/s | 4.8 MB 00:00 2024-09-11T00:08:55.617 INFO:teuthology.orchestra.run.smithi117.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.2 MB/s | 171 kB 00:00 2024-09-11T00:08:55.718 INFO:teuthology.orchestra.run.smithi117.stdout:(28/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 16 MB/s | 3.2 MB 00:00 2024-09-11T00:08:55.744 INFO:teuthology.orchestra.run.smithi117.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 273 kB/s | 34 kB 00:00 2024-09-11T00:08:55.762 INFO:teuthology.orchestra.run.smithi103.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-09-11T00:08:55.793 INFO:teuthology.orchestra.run.smithi117.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.0 MB/s | 154 kB 00:00 2024-09-11T00:08:56.262 INFO:teuthology.orchestra.run.smithi183.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 16 MB/s | 11 MB 00:00 2024-09-11T00:08:56.329 INFO:teuthology.orchestra.run.smithi183.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 379 kB/s | 25 kB 00:00 2024-09-11T00:08:56.569 INFO:teuthology.orchestra.run.smithi117.stdout:(31/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 17 MB/s | 14 MB 00:00 2024-09-11T00:08:56.661 INFO:teuthology.orchestra.run.smithi117.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 3.1 MB/s | 291 kB 00:00 2024-09-11T00:08:56.728 INFO:teuthology.orchestra.run.smithi117.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 740 kB/s | 49 kB 00:00 2024-09-11T00:08:56.794 INFO:teuthology.orchestra.run.smithi117.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 247 kB/s | 16 kB 00:00 2024-09-11T00:08:56.811 INFO:teuthology.orchestra.run.smithi103.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 857 kB/s | 900 kB 00:01 2024-09-11T00:08:56.870 INFO:teuthology.orchestra.run.smithi117.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 3.7 MB/s | 285 kB 00:00 2024-09-11T00:08:56.970 INFO:teuthology.orchestra.run.smithi117.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 7.7 MB/s | 783 kB 00:00 2024-09-11T00:08:56.995 INFO:teuthology.orchestra.run.smithi103.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 791 kB/s | 145 kB 00:00 2024-09-11T00:08:57.037 INFO:teuthology.orchestra.run.smithi117.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 273 kB/s | 18 kB 00:00 2024-09-11T00:08:57.229 INFO:teuthology.orchestra.run.smithi117.stdout:(38/157): ceph-mgr-diskprediction-local-19.3.0- 5.1 MB/s | 7.4 MB 00:01 2024-09-11T00:08:57.254 INFO:teuthology.orchestra.run.smithi117.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 368 kB/s | 80 kB 00:00 2024-09-11T00:08:57.363 INFO:teuthology.orchestra.run.smithi117.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 5.9 MB/s | 801 kB 00:00 2024-09-11T00:08:57.413 INFO:teuthology.orchestra.run.smithi117.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-09-11T00:08:57.439 INFO:teuthology.orchestra.run.smithi117.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-11T00:08:57.489 INFO:teuthology.orchestra.run.smithi117.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-11T00:08:57.514 INFO:teuthology.orchestra.run.smithi117.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-11T00:08:57.539 INFO:teuthology.orchestra.run.smithi117.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-11T00:08:57.573 INFO:teuthology.orchestra.run.smithi117.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-11T00:08:57.606 INFO:teuthology.orchestra.run.smithi117.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-11T00:08:57.813 INFO:teuthology.orchestra.run.smithi183.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 7.0 MB/s | 17 MB 00:02 2024-09-11T00:08:57.873 INFO:teuthology.orchestra.run.smithi117.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-09-11T00:08:57.879 INFO:teuthology.orchestra.run.smithi183.stdout:(11/157): ceph-base-19.3.0-4854.g8f219c47.el9.x 1.3 MB/s | 5.5 MB 00:04 2024-09-11T00:08:57.905 INFO:teuthology.orchestra.run.smithi183.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 2.4 MB/s | 225 kB 00:00 2024-09-11T00:08:57.915 INFO:teuthology.orchestra.run.smithi117.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 943 kB 00:00 2024-09-11T00:08:57.930 INFO:teuthology.orchestra.run.smithi183.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 539 kB/s | 27 kB 00:00 2024-09-11T00:08:57.940 INFO:teuthology.orchestra.run.smithi117.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-11T00:08:57.947 INFO:teuthology.orchestra.run.smithi183.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 761 kB/s | 32 kB 00:00 2024-09-11T00:08:57.997 INFO:teuthology.orchestra.run.smithi183.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 3.2 MB/s | 162 kB 00:00 2024-09-11T00:08:58.056 INFO:teuthology.orchestra.run.smithi183.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 2.1 MB/s | 127 kB 00:00 2024-09-11T00:08:58.156 INFO:teuthology.orchestra.run.smithi183.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 5.0 MB/s | 507 kB 00:00 2024-09-11T00:08:58.221 INFO:teuthology.orchestra.run.smithi103.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-09-11T00:08:58.257 INFO:teuthology.orchestra.run.smithi183.stdout:(18/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 2.3 MB/s | 755 kB 00:00 2024-09-11T00:08:58.429 INFO:teuthology.orchestra.run.smithi103.stdout:(5/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 1.9 MB/s | 5.5 MB 00:02 2024-09-11T00:08:58.491 INFO:teuthology.orchestra.run.smithi117.stdout:(51/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 12 MB/s | 50 MB 00:04 2024-09-11T00:08:58.524 INFO:teuthology.orchestra.run.smithi117.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 36 kB/s | 22 kB 00:00 2024-09-11T00:08:58.550 INFO:teuthology.orchestra.run.smithi117.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 515 kB/s | 313 kB 00:00 2024-09-11T00:08:58.600 INFO:teuthology.orchestra.run.smithi117.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-11T00:08:58.625 INFO:teuthology.orchestra.run.smithi117.stdout:(55/157): flexiblas-3.0.4-8.el9.x86_64.rpm 244 kB/s | 33 kB 00:00 2024-09-11T00:08:58.651 INFO:teuthology.orchestra.run.smithi117.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-11T00:08:58.696 INFO:teuthology.orchestra.run.smithi103.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-09-11T00:08:58.709 INFO:teuthology.orchestra.run.smithi117.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-11T00:08:58.724 INFO:teuthology.orchestra.run.smithi183.stdout:(19/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 21 MB/s | 50 MB 00:02 2024-09-11T00:08:58.740 INFO:teuthology.orchestra.run.smithi183.stdout:(20/157): python3-ceph-argparse-19.3.0-4854.g8f 93 kB/s | 45 kB 00:00 2024-09-11T00:08:58.766 INFO:teuthology.orchestra.run.smithi183.stdout:(21/157): python3-ceph-common-19.3.0-4854.g8f21 3.4 MB/s | 143 kB 00:00 2024-09-11T00:08:58.776 INFO:teuthology.orchestra.run.smithi117.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-09-11T00:08:58.801 INFO:teuthology.orchestra.run.smithi117.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-09-11T00:08:58.836 INFO:teuthology.orchestra.run.smithi117.stdout:(60/157): libnbd-1.20.2-2.el9.x86_64.rpm 809 kB/s | 171 kB 00:00 2024-09-11T00:08:58.858 INFO:teuthology.orchestra.run.smithi183.stdout:(22/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 7.9 MB/s | 5.5 MB 00:00 2024-09-11T00:08:58.878 INFO:teuthology.orchestra.run.smithi117.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 6.3 MB/s | 662 kB 00:00 2024-09-11T00:08:58.883 INFO:teuthology.orchestra.run.smithi183.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 2.7 MB/s | 321 kB 00:00 2024-09-11T00:08:58.895 INFO:teuthology.orchestra.run.smithi117.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-11T00:08:58.908 INFO:teuthology.orchestra.run.smithi183.stdout:(24/157): python3-cephfs-19.3.0-4854.g8f219c47. 965 kB/s | 161 kB 00:00 2024-09-11T00:08:58.929 INFO:teuthology.orchestra.run.smithi117.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-11T00:08:58.933 INFO:teuthology.orchestra.run.smithi183.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 2.0 MB/s | 99 kB 00:00 2024-09-11T00:08:58.946 INFO:teuthology.orchestra.run.smithi117.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-11T00:08:58.959 INFO:teuthology.orchestra.run.smithi183.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-11T00:08:58.984 INFO:teuthology.orchestra.run.smithi183.stdout:(27/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.0 MB/s | 306 kB 00:00 2024-09-11T00:08:58.988 INFO:teuthology.orchestra.run.smithi117.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-11T00:08:59.005 INFO:teuthology.orchestra.run.smithi117.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 657 kB/s | 38 kB 00:00 2024-09-11T00:08:59.026 INFO:teuthology.orchestra.run.smithi183.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 819 kB/s | 34 kB 00:00 2024-09-11T00:08:59.030 INFO:teuthology.orchestra.run.smithi117.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-11T00:08:59.051 INFO:teuthology.orchestra.run.smithi183.stdout:(29/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 1.8 MB/s | 171 kB 00:00 2024-09-11T00:08:59.076 INFO:teuthology.orchestra.run.smithi183.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.0 MB/s | 154 kB 00:00 2024-09-11T00:08:59.127 INFO:teuthology.orchestra.run.smithi183.stdout:(31/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 16 MB/s | 3.2 MB 00:00 2024-09-11T00:08:59.155 INFO:teuthology.orchestra.run.smithi117.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-09-11T00:08:59.177 INFO:teuthology.orchestra.run.smithi183.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 5.7 MB/s | 291 kB 00:00 2024-09-11T00:08:59.227 INFO:teuthology.orchestra.run.smithi183.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 990 kB/s | 49 kB 00:00 2024-09-11T00:08:59.230 INFO:teuthology.orchestra.run.smithi117.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 20 MB/s | 4.9 MB 00:00 2024-09-11T00:08:59.272 INFO:teuthology.orchestra.run.smithi117.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-09-11T00:08:59.278 INFO:teuthology.orchestra.run.smithi183.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 328 kB/s | 16 kB 00:00 2024-09-11T00:08:59.306 INFO:teuthology.orchestra.run.smithi117.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-11T00:08:59.328 INFO:teuthology.orchestra.run.smithi183.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.6 MB/s | 285 kB 00:00 2024-09-11T00:08:59.348 INFO:teuthology.orchestra.run.smithi117.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-11T00:08:59.388 INFO:teuthology.orchestra.run.smithi103.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 5.0 MB/s | 4.8 MB 00:00 2024-09-11T00:08:59.390 INFO:teuthology.orchestra.run.smithi117.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-11T00:08:59.395 INFO:teuthology.orchestra.run.smithi183.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 12 MB/s | 783 kB 00:00 2024-09-11T00:08:59.415 INFO:teuthology.orchestra.run.smithi117.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-11T00:08:59.445 INFO:teuthology.orchestra.run.smithi183.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 366 kB/s | 18 kB 00:00 2024-09-11T00:08:59.482 INFO:teuthology.orchestra.run.smithi117.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 919 kB/s | 61 kB 00:00 2024-09-11T00:08:59.549 INFO:teuthology.orchestra.run.smithi117.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-11T00:08:59.554 INFO:teuthology.orchestra.run.smithi183.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 738 kB/s | 80 kB 00:00 2024-09-11T00:08:59.575 INFO:teuthology.orchestra.run.smithi117.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-11T00:08:59.608 INFO:teuthology.orchestra.run.smithi117.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-11T00:08:59.636 INFO:teuthology.orchestra.run.smithi117.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-11T00:08:59.646 INFO:teuthology.orchestra.run.smithi183.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-11T00:08:59.709 INFO:teuthology.orchestra.run.smithi117.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-09-11T00:08:59.729 INFO:teuthology.orchestra.run.smithi183.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-11T00:08:59.734 INFO:teuthology.orchestra.run.smithi117.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-11T00:08:59.760 INFO:teuthology.orchestra.run.smithi117.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T00:08:59.771 INFO:teuthology.orchestra.run.smithi183.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-09-11T00:08:59.784 INFO:teuthology.orchestra.run.smithi117.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-09-11T00:08:59.796 INFO:teuthology.orchestra.run.smithi183.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-11T00:08:59.826 INFO:teuthology.orchestra.run.smithi117.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-11T00:08:59.840 INFO:teuthology.orchestra.run.smithi183.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-11T00:08:59.852 INFO:teuthology.orchestra.run.smithi117.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-11T00:08:59.872 INFO:teuthology.orchestra.run.smithi183.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-09-11T00:08:59.897 INFO:teuthology.orchestra.run.smithi183.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-11T00:08:59.918 INFO:teuthology.orchestra.run.smithi117.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-09-11T00:08:59.922 INFO:teuthology.orchestra.run.smithi183.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-11T00:08:59.944 INFO:teuthology.orchestra.run.smithi117.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 658 kB/s | 77 kB 00:00 2024-09-11T00:08:59.948 INFO:teuthology.orchestra.run.smithi183.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-11T00:08:59.969 INFO:teuthology.orchestra.run.smithi117.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 360 kB/s | 42 kB 00:00 2024-09-11T00:08:59.994 INFO:teuthology.orchestra.run.smithi117.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-11T00:09:00.020 INFO:teuthology.orchestra.run.smithi117.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-09-11T00:09:00.045 INFO:teuthology.orchestra.run.smithi117.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-11T00:09:00.070 INFO:teuthology.orchestra.run.smithi117.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 672 kB/s | 51 kB 00:00 2024-09-11T00:09:00.096 INFO:teuthology.orchestra.run.smithi117.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-11T00:09:00.129 INFO:teuthology.orchestra.run.smithi117.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-09-11T00:09:00.154 INFO:teuthology.orchestra.run.smithi117.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-09-11T00:09:00.188 INFO:teuthology.orchestra.run.smithi117.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T00:09:00.230 INFO:teuthology.orchestra.run.smithi117.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 476 kB/s | 36 kB 00:00 2024-09-11T00:09:00.256 INFO:teuthology.orchestra.run.smithi117.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-09-11T00:09:00.289 INFO:teuthology.orchestra.run.smithi117.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-11T00:09:00.306 INFO:teuthology.orchestra.run.smithi117.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T00:09:00.340 INFO:teuthology.orchestra.run.smithi117.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-11T00:09:00.356 INFO:teuthology.orchestra.run.smithi117.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-11T00:09:00.523 INFO:teuthology.orchestra.run.smithi117.stdout:(103/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-11T00:09:00.540 INFO:teuthology.orchestra.run.smithi117.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-09-11T00:09:00.557 INFO:teuthology.orchestra.run.smithi117.stdout:(105/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 206 kB/s | 45 kB 00:00 2024-09-11T00:09:00.607 INFO:teuthology.orchestra.run.smithi117.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-09-11T00:09:00.661 INFO:teuthology.orchestra.run.smithi117.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-09-11T00:09:00.682 INFO:teuthology.orchestra.run.smithi183.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 908 kB/s | 689 kB 00:00 2024-09-11T00:09:00.683 INFO:teuthology.orchestra.run.smithi117.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-09-11T00:09:00.708 INFO:teuthology.orchestra.run.smithi117.stdout:(109/157): grpc-data-1.46.7-10.el9.noarch.rpm 197 kB/s | 19 kB 00:00 2024-09-11T00:09:00.734 INFO:teuthology.orchestra.run.smithi117.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-09-11T00:09:00.759 INFO:teuthology.orchestra.run.smithi117.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 965 kB/s | 49 kB 00:00 2024-09-11T00:09:00.784 INFO:teuthology.orchestra.run.smithi117.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-11T00:09:00.810 INFO:teuthology.orchestra.run.smithi117.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-11T00:09:00.823 INFO:teuthology.orchestra.run.smithi183.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-11T00:09:00.849 INFO:teuthology.orchestra.run.smithi183.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-11T00:09:00.852 INFO:teuthology.orchestra.run.smithi117.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-11T00:09:00.877 INFO:teuthology.orchestra.run.smithi117.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T00:09:00.910 INFO:teuthology.orchestra.run.smithi117.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-09-11T00:09:00.932 INFO:teuthology.orchestra.run.smithi183.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-11T00:09:00.936 INFO:teuthology.orchestra.run.smithi117.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-09-11T00:09:00.961 INFO:teuthology.orchestra.run.smithi117.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-11T00:09:00.963 INFO:teuthology.orchestra.run.smithi103.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 6.9 MB/s | 11 MB 00:01 2024-09-11T00:09:00.966 INFO:teuthology.orchestra.run.smithi183.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-11T00:09:00.986 INFO:teuthology.orchestra.run.smithi117.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T00:09:01.020 INFO:teuthology.orchestra.run.smithi117.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-11T00:09:01.033 INFO:teuthology.orchestra.run.smithi183.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-11T00:09:01.045 INFO:teuthology.orchestra.run.smithi117.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-11T00:09:01.056 INFO:teuthology.orchestra.run.smithi103.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 276 kB/s | 25 kB 00:00 2024-09-11T00:09:01.070 INFO:teuthology.orchestra.run.smithi117.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T00:09:01.104 INFO:teuthology.orchestra.run.smithi117.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-11T00:09:01.129 INFO:teuthology.orchestra.run.smithi117.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-11T00:09:01.141 INFO:teuthology.orchestra.run.smithi183.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-11T00:09:01.196 INFO:teuthology.orchestra.run.smithi117.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 4.4 MB 00:00 2024-09-11T00:09:01.222 INFO:teuthology.orchestra.run.smithi117.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-09-11T00:09:01.247 INFO:teuthology.orchestra.run.smithi117.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-11T00:09:01.258 INFO:teuthology.orchestra.run.smithi183.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-09-11T00:09:01.272 INFO:teuthology.orchestra.run.smithi117.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-11T00:09:01.289 INFO:teuthology.orchestra.run.smithi117.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 468 kB/s | 20 kB 00:00 2024-09-11T00:09:01.306 INFO:teuthology.orchestra.run.smithi117.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-11T00:09:01.331 INFO:teuthology.orchestra.run.smithi117.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T00:09:01.348 INFO:teuthology.orchestra.run.smithi117.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-11T00:09:01.375 INFO:teuthology.orchestra.run.smithi183.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-11T00:09:01.382 INFO:teuthology.orchestra.run.smithi117.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 9.3 MB/s | 1.0 MB 00:00 2024-09-11T00:09:01.407 INFO:teuthology.orchestra.run.smithi117.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-09-11T00:09:01.424 INFO:teuthology.orchestra.run.smithi117.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 392 kB/s | 16 kB 00:00 2024-09-11T00:09:01.466 INFO:teuthology.orchestra.run.smithi117.stdout:(136/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-11T00:09:01.467 INFO:teuthology.orchestra.run.smithi183.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-09-11T00:09:01.491 INFO:teuthology.orchestra.run.smithi117.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-09-11T00:09:01.492 INFO:teuthology.orchestra.run.smithi183.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-11T00:09:01.517 INFO:teuthology.orchestra.run.smithi117.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-11T00:09:01.542 INFO:teuthology.orchestra.run.smithi117.stdout:(139/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-09-11T00:09:01.567 INFO:teuthology.orchestra.run.smithi117.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-11T00:09:01.576 INFO:teuthology.orchestra.run.smithi183.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-11T00:09:01.592 INFO:teuthology.orchestra.run.smithi117.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-11T00:09:01.626 INFO:teuthology.orchestra.run.smithi117.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-11T00:09:01.651 INFO:teuthology.orchestra.run.smithi117.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 888 kB/s | 74 kB 00:00 2024-09-11T00:09:01.664 INFO:teuthology.orchestra.run.smithi103.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 5.8 MB/s | 17 MB 00:02 2024-09-11T00:09:01.677 INFO:teuthology.orchestra.run.smithi117.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T00:09:01.702 INFO:teuthology.orchestra.run.smithi117.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-11T00:09:01.727 INFO:teuthology.orchestra.run.smithi117.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-11T00:09:01.752 INFO:teuthology.orchestra.run.smithi183.stdout:(60/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 5.2 MB/s | 14 MB 00:02 2024-09-11T00:09:01.752 INFO:teuthology.orchestra.run.smithi117.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T00:09:01.755 INFO:teuthology.orchestra.run.smithi103.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 2.4 MB/s | 225 kB 00:00 2024-09-11T00:09:01.776 INFO:teuthology.orchestra.run.smithi183.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-11T00:09:01.779 INFO:teuthology.orchestra.run.smithi117.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-11T00:09:01.802 INFO:teuthology.orchestra.run.smithi183.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 2.5 MB/s | 581 kB 00:00 2024-09-11T00:09:01.823 INFO:teuthology.orchestra.run.smithi103.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 406 kB/s | 27 kB 00:00 2024-09-11T00:09:01.828 INFO:teuthology.orchestra.run.smithi117.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T00:09:01.885 INFO:teuthology.orchestra.run.smithi183.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-11T00:09:01.892 INFO:teuthology.orchestra.run.smithi103.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 479 kB/s | 32 kB 00:00 2024-09-11T00:09:01.912 INFO:teuthology.orchestra.run.smithi117.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 28 MB/s | 2.3 MB 00:00 2024-09-11T00:09:01.919 INFO:teuthology.orchestra.run.smithi183.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-11T00:09:01.987 INFO:teuthology.orchestra.run.smithi117.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-11T00:09:01.995 INFO:teuthology.orchestra.run.smithi183.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-11T00:09:02.028 INFO:teuthology.orchestra.run.smithi183.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 355 kB/s | 38 kB 00:00 2024-09-11T00:09:02.029 INFO:teuthology.orchestra.run.smithi117.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 982 kB/s | 41 kB 00:00 2024-09-11T00:09:02.054 INFO:teuthology.orchestra.run.smithi183.stdout:(67/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 81 kB/s | 24 kB 00:00 2024-09-11T00:09:02.088 INFO:teuthology.orchestra.run.smithi117.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-11T00:09:02.138 INFO:teuthology.orchestra.run.smithi117.stdout:(154/157): librados2-19.3.0-4854.g8f219c47.el9. 9.0 MB/s | 3.5 MB 00:00 2024-09-11T00:09:02.140 INFO:teuthology.orchestra.run.smithi103.stdout:(14/157): ceph-common-19.3.0-4854.g8f219c47.el9 3.3 MB/s | 22 MB 00:06 2024-09-11T00:09:02.180 INFO:teuthology.orchestra.run.smithi117.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-11T00:09:02.199 INFO:teuthology.orchestra.run.smithi103.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 2.4 MB/s | 755 kB 00:00 2024-09-11T00:09:02.238 INFO:teuthology.orchestra.run.smithi117.stdout:(156/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 7.0 MB/s | 3.2 MB 00:00 2024-09-11T00:09:02.266 INFO:teuthology.orchestra.run.smithi103.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.3 MB/s | 162 kB 00:00 2024-09-11T00:09:02.299 INFO:teuthology.orchestra.run.smithi103.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 1.2 MB/s | 127 kB 00:00 2024-09-11T00:09:02.347 INFO:teuthology.orchestra.run.smithi117.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-09-11T00:09:02.353 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:09:02.353 INFO:teuthology.orchestra.run.smithi117.stdout:Total 24 MB/s | 240 MB 00:09 2024-09-11T00:09:02.366 INFO:teuthology.orchestra.run.smithi103.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 5.0 MB/s | 507 kB 00:00 2024-09-11T00:09:02.433 INFO:teuthology.orchestra.run.smithi103.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 678 kB/s | 45 kB 00:00 2024-09-11T00:09:02.492 INFO:teuthology.orchestra.run.smithi103.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.4 MB/s | 143 kB 00:00 2024-09-11T00:09:02.542 INFO:teuthology.orchestra.run.smithi103.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 3.2 MB/s | 161 kB 00:00 2024-09-11T00:09:02.617 INFO:teuthology.orchestra.run.smithi103.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 4.2 MB/s | 321 kB 00:00 2024-09-11T00:09:02.685 INFO:teuthology.orchestra.run.smithi103.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 4.5 MB/s | 306 kB 00:00 2024-09-11T00:09:02.735 INFO:teuthology.orchestra.run.smithi103.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 2.0 MB/s | 99 kB 00:00 2024-09-11T00:09:02.787 INFO:teuthology.orchestra.run.smithi183.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-09-11T00:09:02.810 INFO:teuthology.orchestra.run.smithi103.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-11T00:09:02.912 INFO:teuthology.orchestra.run.smithi183.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-11T00:09:03.046 INFO:teuthology.orchestra.run.smithi183.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-11T00:09:03.111 INFO:teuthology.orchestra.run.smithi103.stdout:(26/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 6.8 MB/s | 5.5 MB 00:00 2024-09-11T00:09:03.186 INFO:teuthology.orchestra.run.smithi103.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.2 MB/s | 171 kB 00:00 2024-09-11T00:09:03.253 INFO:teuthology.orchestra.run.smithi103.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 515 kB/s | 34 kB 00:00 2024-09-11T00:09:03.311 INFO:teuthology.orchestra.run.smithi103.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.6 MB/s | 154 kB 00:00 2024-09-11T00:09:03.370 INFO:teuthology.orchestra.run.smithi103.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 5.7 MB/s | 3.2 MB 00:00 2024-09-11T00:09:03.590 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-11T00:09:03.742 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-11T00:09:03.743 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-11T00:09:04.253 INFO:teuthology.orchestra.run.smithi103.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 8.4 MB/s | 7.4 MB 00:00 2024-09-11T00:09:04.320 INFO:teuthology.orchestra.run.smithi103.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.3 MB/s | 291 kB 00:00 2024-09-11T00:09:04.396 INFO:teuthology.orchestra.run.smithi103.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 659 kB/s | 49 kB 00:00 2024-09-11T00:09:04.471 INFO:teuthology.orchestra.run.smithi103.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 219 kB/s | 16 kB 00:00 2024-09-11T00:09:04.563 INFO:teuthology.orchestra.run.smithi103.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 3.1 MB/s | 285 kB 00:00 2024-09-11T00:09:04.680 INFO:teuthology.orchestra.run.smithi103.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 6.6 MB/s | 783 kB 00:00 2024-09-11T00:09:04.756 INFO:teuthology.orchestra.run.smithi103.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 244 kB/s | 18 kB 00:00 2024-09-11T00:09:04.906 INFO:teuthology.orchestra.run.smithi103.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 533 kB/s | 80 kB 00:00 2024-09-11T00:09:05.114 INFO:teuthology.orchestra.run.smithi103.stdout:(39/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 7.9 MB/s | 14 MB 00:01 2024-09-11T00:09:05.239 INFO:teuthology.orchestra.run.smithi103.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-11T00:09:05.299 INFO:teuthology.orchestra.run.smithi103.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-11T00:09:05.324 INFO:teuthology.orchestra.run.smithi103.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 916 kB/s | 191 kB 00:00 2024-09-11T00:09:05.383 INFO:teuthology.orchestra.run.smithi103.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-11T00:09:05.424 INFO:teuthology.orchestra.run.smithi103.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-09-11T00:09:05.450 INFO:teuthology.orchestra.run.smithi103.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-11T00:09:05.475 INFO:teuthology.orchestra.run.smithi103.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-11T00:09:05.517 INFO:teuthology.orchestra.run.smithi103.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-11T00:09:05.619 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-11T00:09:05.619 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-11T00:09:05.746 INFO:teuthology.orchestra.run.smithi183.stdout:(71/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.2 MB/s | 4.4 MB 00:03 2024-09-11T00:09:05.954 INFO:teuthology.orchestra.run.smithi183.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 1.2 MB/s | 4.9 MB 00:03 2024-09-11T00:09:06.154 INFO:teuthology.orchestra.run.smithi183.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 438 kB/s | 87 kB 00:00 2024-09-11T00:09:06.309 INFO:teuthology.orchestra.run.smithi103.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 827 kB/s | 689 kB 00:00 2024-09-11T00:09:06.434 INFO:teuthology.orchestra.run.smithi103.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 857 kB/s | 107 kB 00:00 2024-09-11T00:09:06.960 INFO:teuthology.orchestra.run.smithi103.stdout:(50/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 8.5 MB/s | 50 MB 00:05 2024-09-11T00:09:06.994 INFO:teuthology.orchestra.run.smithi103.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-11T00:09:07.028 INFO:teuthology.orchestra.run.smithi103.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 625 kB/s | 943 kB 00:01 2024-09-11T00:09:07.103 INFO:teuthology.orchestra.run.smithi103.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-11T00:09:07.211 INFO:teuthology.orchestra.run.smithi103.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-11T00:09:07.328 INFO:teuthology.orchestra.run.smithi103.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-09-11T00:09:07.453 INFO:teuthology.orchestra.run.smithi103.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.3 MB/s | 171 kB 00:00 2024-09-11T00:09:07.554 INFO:teuthology.orchestra.run.smithi183.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 1.3 MB/s | 5.8 MB 00:04 2024-09-11T00:09:07.570 INFO:teuthology.orchestra.run.smithi103.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-11T00:09:07.588 INFO:teuthology.orchestra.run.smithi183.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 561 kB/s | 1.0 MB 00:01 2024-09-11T00:09:07.672 INFO:teuthology.orchestra.run.smithi183.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 524 kB/s | 61 kB 00:00 2024-09-11T00:09:07.687 INFO:teuthology.orchestra.run.smithi103.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 389 kB/s | 45 kB 00:00 2024-09-11T00:09:07.705 INFO:teuthology.orchestra.run.smithi183.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-11T00:09:07.713 INFO:teuthology.orchestra.run.smithi103.stdout:(59/157): flex-2.6.4-9.el9.x86_64.rpm 417 kB/s | 313 kB 00:00 2024-09-11T00:09:07.790 INFO:teuthology.orchestra.run.smithi183.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.0 MB/s | 248 kB 00:00 2024-09-11T00:09:07.816 INFO:teuthology.orchestra.run.smithi183.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 233 kB/s | 26 kB 00:00 2024-09-11T00:09:07.896 INFO:teuthology.orchestra.run.smithi103.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-09-11T00:09:07.907 INFO:teuthology.orchestra.run.smithi183.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-09-11T00:09:07.933 INFO:teuthology.orchestra.run.smithi183.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-11T00:09:07.938 INFO:teuthology.orchestra.run.smithi103.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 2.5 MB/s | 581 kB 00:00 2024-09-11T00:09:08.013 INFO:teuthology.orchestra.run.smithi103.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 209 kB/s | 24 kB 00:00 2024-09-11T00:09:08.016 INFO:teuthology.orchestra.run.smithi183.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-11T00:09:08.064 INFO:teuthology.orchestra.run.smithi103.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 379 kB/s | 47 kB 00:00 2024-09-11T00:09:08.139 INFO:teuthology.orchestra.run.smithi103.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-11T00:09:08.142 INFO:teuthology.orchestra.run.smithi183.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-11T00:09:08.181 INFO:teuthology.orchestra.run.smithi103.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-11T00:09:08.225 INFO:teuthology.orchestra.run.smithi183.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-11T00:09:08.248 INFO:teuthology.orchestra.run.smithi103.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 353 kB/s | 38 kB 00:00 2024-09-11T00:09:08.259 INFO:teuthology.orchestra.run.smithi183.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-11T00:09:08.334 INFO:teuthology.orchestra.run.smithi183.stdout:(86/157): python3-babel-2.9.1-2.el9.noarch.rpm 2.7 MB/s | 6.0 MB 00:02 2024-09-11T00:09:08.360 INFO:teuthology.orchestra.run.smithi183.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 577 kB/s | 77 kB 00:00 2024-09-11T00:09:08.385 INFO:teuthology.orchestra.run.smithi183.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-11T00:09:08.452 INFO:teuthology.orchestra.run.smithi183.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-11T00:09:08.468 INFO:teuthology.orchestra.run.smithi183.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-11T00:09:08.502 INFO:teuthology.orchestra.run.smithi183.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-11T00:09:08.586 INFO:teuthology.orchestra.run.smithi183.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 381 kB/s | 51 kB 00:00 2024-09-11T00:09:08.590 INFO:teuthology.orchestra.run.smithi103.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-11T00:09:08.620 INFO:teuthology.orchestra.run.smithi183.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 355 kB/s | 54 kB 00:00 2024-09-11T00:09:08.695 INFO:teuthology.orchestra.run.smithi183.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 382 kB/s | 42 kB 00:00 2024-09-11T00:09:08.699 INFO:teuthology.orchestra.run.smithi103.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 9.5 MB/s | 4.9 MB 00:00 2024-09-11T00:09:08.737 INFO:teuthology.orchestra.run.smithi183.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-11T00:09:08.749 INFO:teuthology.orchestra.run.smithi103.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-11T00:09:08.812 INFO:teuthology.orchestra.run.smithi183.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-11T00:09:08.824 INFO:teuthology.orchestra.run.smithi103.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-11T00:09:08.846 INFO:teuthology.orchestra.run.smithi183.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 330 kB/s | 36 kB 00:00 2024-09-11T00:09:08.866 INFO:teuthology.orchestra.run.smithi103.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-11T00:09:08.979 INFO:teuthology.orchestra.run.smithi183.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 284 kB/s | 47 kB 00:00 2024-09-11T00:09:09.013 INFO:teuthology.orchestra.run.smithi183.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-09-11T00:09:09.039 INFO:teuthology.orchestra.run.smithi103.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2024-09-11T00:09:09.088 INFO:teuthology.orchestra.run.smithi183.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 533 kB/s | 58 kB 00:00 2024-09-11T00:09:09.122 INFO:teuthology.orchestra.run.smithi183.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 883 kB/s | 96 kB 00:00 2024-09-11T00:09:09.125 INFO:teuthology.orchestra.run.smithi103.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-09-11T00:09:09.159 INFO:teuthology.orchestra.run.smithi103.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 729 kB/s | 87 kB 00:00 2024-09-11T00:09:09.205 INFO:teuthology.orchestra.run.smithi183.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 544 kB/s | 64 kB 00:00 2024-09-11T00:09:09.209 INFO:teuthology.orchestra.run.smithi103.stdout:(75/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.4 MB/s | 3.0 MB 00:02 2024-09-11T00:09:09.239 INFO:teuthology.orchestra.run.smithi183.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 383 kB/s | 45 kB 00:00 2024-09-11T00:09:09.276 INFO:teuthology.orchestra.run.smithi103.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 523 kB/s | 61 kB 00:00 2024-09-11T00:09:09.406 INFO:teuthology.orchestra.run.smithi183.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-11T00:09:09.418 INFO:teuthology.orchestra.run.smithi103.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-09-11T00:09:09.431 INFO:teuthology.orchestra.run.smithi183.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 99 kB/s | 22 kB 00:00 2024-09-11T00:09:09.443 INFO:teuthology.orchestra.run.smithi103.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-09-11T00:09:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-09-11T00:09:09.481 INFO:teuthology.orchestra.run.smithi183.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.5 MB/s | 863 kB 00:00 2024-09-11T00:09:09.523 INFO:teuthology.orchestra.run.smithi183.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-09-11T00:09:09.527 INFO:teuthology.orchestra.run.smithi103.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 235 kB/s | 26 kB 00:00 2024-09-11T00:09:09.549 INFO:teuthology.orchestra.run.smithi183.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-09-11T00:09:09.553 INFO:teuthology.orchestra.run.smithi103.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-11T00:09:09.574 INFO:teuthology.orchestra.run.smithi183.stdout:(109/157): grpc-data-1.46.7-10.el9.noarch.rpm 215 kB/s | 19 kB 00:00 2024-09-11T00:09:09.586 INFO:teuthology.orchestra.run.smithi103.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-11T00:09:09.599 INFO:teuthology.orchestra.run.smithi183.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-09-11T00:09:09.624 INFO:teuthology.orchestra.run.smithi183.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-09-11T00:09:09.637 INFO:teuthology.orchestra.run.smithi103.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-09-11T00:09:09.650 INFO:teuthology.orchestra.run.smithi183.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-11T00:09:09.675 INFO:teuthology.orchestra.run.smithi183.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-11T00:09:09.703 INFO:teuthology.orchestra.run.smithi103.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-11T00:09:09.717 INFO:teuthology.orchestra.run.smithi183.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-11T00:09:09.751 INFO:teuthology.orchestra.run.smithi183.stdout:(115/157): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-11T00:09:09.762 INFO:teuthology.orchestra.run.smithi103.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-09-11T00:09:09.776 INFO:teuthology.orchestra.run.smithi183.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-09-11T00:09:09.801 INFO:teuthology.orchestra.run.smithi183.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-09-11T00:09:09.812 INFO:teuthology.orchestra.run.smithi103.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 709 kB/s | 77 kB 00:00 2024-09-11T00:09:09.826 INFO:teuthology.orchestra.run.smithi183.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-11T00:09:09.852 INFO:teuthology.orchestra.run.smithi183.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T00:09:09.879 INFO:teuthology.orchestra.run.smithi103.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-09-11T00:09:09.885 INFO:teuthology.orchestra.run.smithi183.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T00:09:09.919 INFO:teuthology.orchestra.run.smithi183.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-11T00:09:09.930 INFO:teuthology.orchestra.run.smithi103.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-09-11T00:09:09.953 INFO:teuthology.orchestra.run.smithi183.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-11T00:09:09.986 INFO:teuthology.orchestra.run.smithi183.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-11T00:09:09.997 INFO:teuthology.orchestra.run.smithi103.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-11T00:09:10.012 INFO:teuthology.orchestra.run.smithi183.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-11T00:09:10.037 INFO:teuthology.orchestra.run.smithi183.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-11T00:09:10.047 INFO:teuthology.orchestra.run.smithi103.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-11T00:09:10.062 INFO:teuthology.orchestra.run.smithi183.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-11T00:09:10.087 INFO:teuthology.orchestra.run.smithi183.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-11T00:09:10.113 INFO:teuthology.orchestra.run.smithi183.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-11T00:09:10.114 INFO:teuthology.orchestra.run.smithi103.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-09-11T00:09:10.164 INFO:teuthology.orchestra.run.smithi103.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 459 kB/s | 54 kB 00:00 2024-09-11T00:09:10.196 INFO:teuthology.orchestra.run.smithi183.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-09-11T00:09:10.222 INFO:teuthology.orchestra.run.smithi183.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T00:09:10.247 INFO:teuthology.orchestra.run.smithi183.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T00:09:10.280 INFO:teuthology.orchestra.run.smithi183.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-11T00:09:10.281 INFO:teuthology.orchestra.run.smithi103.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-09-11T00:09:10.306 INFO:teuthology.orchestra.run.smithi183.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-11T00:09:10.331 INFO:teuthology.orchestra.run.smithi183.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-11T00:09:10.373 INFO:teuthology.orchestra.run.smithi183.stdout:(135/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-09-11T00:09:10.398 INFO:teuthology.orchestra.run.smithi183.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-11T00:09:10.423 INFO:teuthology.orchestra.run.smithi183.stdout:(137/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-11T00:09:10.436 INFO:teuthology.orchestra.run.smithi103.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 285 kB/s | 44 kB 00:00 2024-09-11T00:09:10.449 INFO:teuthology.orchestra.run.smithi183.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-11T00:09:10.476 INFO:teuthology.orchestra.run.smithi183.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2024-09-11T00:09:10.499 INFO:teuthology.orchestra.run.smithi183.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-11T00:09:10.524 INFO:teuthology.orchestra.run.smithi183.stdout:(141/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-11T00:09:10.550 INFO:teuthology.orchestra.run.smithi183.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-11T00:09:10.557 INFO:teuthology.orchestra.run.smithi103.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-11T00:09:10.575 INFO:teuthology.orchestra.run.smithi183.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-11T00:09:10.600 INFO:teuthology.orchestra.run.smithi183.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T00:09:10.617 INFO:teuthology.orchestra.run.smithi183.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-11T00:09:10.642 INFO:teuthology.orchestra.run.smithi183.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-11T00:09:10.665 INFO:teuthology.orchestra.run.smithi103.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 332 kB/s | 36 kB 00:00 2024-09-11T00:09:10.668 INFO:teuthology.orchestra.run.smithi183.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T00:09:10.693 INFO:teuthology.orchestra.run.smithi183.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-11T00:09:10.743 INFO:teuthology.orchestra.run.smithi183.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-11T00:09:10.858 INFO:teuthology.orchestra.run.smithi103.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-09-11T00:09:10.883 INFO:teuthology.orchestra.run.smithi103.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 218 kB/s | 47 kB 00:00 2024-09-11T00:09:10.974 INFO:teuthology.orchestra.run.smithi103.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.6 MB/s | 304 kB 00:00 2024-09-11T00:09:11.000 INFO:teuthology.orchestra.run.smithi103.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 494 kB/s | 58 kB 00:00 2024-09-11T00:09:11.002 INFO:teuthology.orchestra.run.smithi183.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 10 MB/s | 3.5 MB 00:00 2024-09-11T00:09:11.084 INFO:teuthology.orchestra.run.smithi103.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 883 kB/s | 96 kB 00:00 2024-09-11T00:09:11.100 INFO:teuthology.orchestra.run.smithi103.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-09-11T00:09:11.102 INFO:teuthology.orchestra.run.smithi183.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 6.5 MB/s | 2.3 MB 00:00 2024-09-11T00:09:11.144 INFO:teuthology.orchestra.run.smithi183.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 981 kB/s | 41 kB 00:00 2024-09-11T00:09:11.186 INFO:teuthology.orchestra.run.smithi183.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 5.8 MB/s | 249 kB 00:00 2024-09-11T00:09:11.192 INFO:teuthology.orchestra.run.smithi103.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-09-11T00:09:11.209 INFO:teuthology.orchestra.run.smithi103.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 357 kB/s | 45 kB 00:00 2024-09-11T00:09:11.285 INFO:teuthology.orchestra.run.smithi103.stdout:(105/157): python3-numpy-1.20.1-5.el9.x86_64.rp 2.9 MB/s | 5.1 MB 00:01 2024-09-11T00:09:11.320 INFO:teuthology.orchestra.run.smithi183.stdout:(154/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 5.2 MB/s | 3.2 MB 00:00 2024-09-11T00:09:11.344 INFO:teuthology.orchestra.run.smithi103.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 4.0 MB/s | 548 kB 00:00 2024-09-11T00:09:11.369 INFO:teuthology.orchestra.run.smithi103.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-11T00:09:11.387 INFO:teuthology.orchestra.run.smithi183.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 6.8 MB/s | 1.4 MB 00:00 2024-09-11T00:09:11.394 INFO:teuthology.orchestra.run.smithi103.stdout:(108/157): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-09-11T00:09:11.419 INFO:teuthology.orchestra.run.smithi103.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T00:09:11.428 INFO:teuthology.orchestra.run.smithi183.stdout:(156/157): NetworkManager-libnm-1.49.90-1.el9.x 4.2 MB/s | 1.8 MB 00:00 2024-09-11T00:09:11.579 INFO:teuthology.orchestra.run.smithi103.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-09-11T00:09:11.587 INFO:teuthology.orchestra.run.smithi183.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-09-11T00:09:11.593 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:09:11.593 INFO:teuthology.orchestra.run.smithi183.stdout:Total 13 MB/s | 240 MB 00:18 2024-09-11T00:09:11.603 INFO:teuthology.orchestra.run.smithi103.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 264 kB/s | 49 kB 00:00 2024-09-11T00:09:11.646 INFO:teuthology.orchestra.run.smithi103.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.7 MB/s | 151 kB 00:00 2024-09-11T00:09:11.671 INFO:teuthology.orchestra.run.smithi103.stdout:(113/157): libunwind-1.6.2-1.el9.x86_64.rpm 737 kB/s | 67 kB 00:00 2024-09-11T00:09:11.713 INFO:teuthology.orchestra.run.smithi103.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-09-11T00:09:11.738 INFO:teuthology.orchestra.run.smithi103.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T00:09:11.763 INFO:teuthology.orchestra.run.smithi103.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-11T00:09:11.789 INFO:teuthology.orchestra.run.smithi103.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 573 kB/s | 14 kB 00:00 2024-09-11T00:09:11.814 INFO:teuthology.orchestra.run.smithi103.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.7 MB/s | 548 kB 00:00 2024-09-11T00:09:11.839 INFO:teuthology.orchestra.run.smithi103.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T00:09:11.873 INFO:teuthology.orchestra.run.smithi103.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.8 MB/s | 303 kB 00:00 2024-09-11T00:09:11.924 INFO:teuthology.orchestra.run.smithi103.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-09-11T00:09:11.928 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-11T00:09:11.957 INFO:teuthology.orchestra.run.smithi103.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-11T00:09:11.991 INFO:teuthology.orchestra.run.smithi103.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-11T00:09:12.016 INFO:teuthology.orchestra.run.smithi103.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-11T00:09:12.041 INFO:teuthology.orchestra.run.smithi103.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-11T00:09:12.050 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-11T00:09:12.067 INFO:teuthology.orchestra.run.smithi103.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 935 kB/s | 23 kB 00:00 2024-09-11T00:09:12.092 INFO:teuthology.orchestra.run.smithi103.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-11T00:09:12.117 INFO:teuthology.orchestra.run.smithi103.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-11T00:09:12.174 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-11T00:09:12.201 INFO:teuthology.orchestra.run.smithi103.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-09-11T00:09:12.257 INFO:teuthology.orchestra.run.smithi103.stdout:(130/157): python3-kubernetes-26.1.0-2.el9.noar 7.3 MB/s | 1.0 MB 00:00 2024-09-11T00:09:12.285 INFO:teuthology.orchestra.run.smithi103.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 555 kB/s | 46 kB 00:00 2024-09-11T00:09:12.310 INFO:teuthology.orchestra.run.smithi103.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-11T00:09:12.327 INFO:teuthology.orchestra.run.smithi103.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-09-11T00:09:12.344 INFO:teuthology.orchestra.run.smithi103.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 991 kB/s | 16 kB 00:00 2024-09-11T00:09:12.352 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-11T00:09:12.393 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-11T00:09:12.400 INFO:teuthology.orchestra.run.smithi103.stdout:(135/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-09-11T00:09:12.425 INFO:teuthology.orchestra.run.smithi103.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-09-11T00:09:12.426 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-11T00:09:12.450 INFO:teuthology.orchestra.run.smithi103.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-11T00:09:12.478 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-11T00:09:12.543 INFO:teuthology.orchestra.run.smithi103.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 263 kB/s | 31 kB 00:00 2024-09-11T00:09:12.568 INFO:teuthology.orchestra.run.smithi103.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-09-11T00:09:12.610 INFO:teuthology.orchestra.run.smithi103.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-09-11T00:09:12.635 INFO:teuthology.orchestra.run.smithi103.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 534 kB/s | 36 kB 00:00 2024-09-11T00:09:12.660 INFO:teuthology.orchestra.run.smithi103.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 900 kB/s | 188 kB 00:00 2024-09-11T00:09:12.685 INFO:teuthology.orchestra.run.smithi103.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 989 kB/s | 74 kB 00:00 2024-09-11T00:09:12.711 INFO:teuthology.orchestra.run.smithi103.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T00:09:12.728 INFO:teuthology.orchestra.run.smithi103.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-11T00:09:12.753 INFO:teuthology.orchestra.run.smithi103.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-11T00:09:12.777 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-11T00:09:12.787 INFO:teuthology.orchestra.run.smithi103.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-11T00:09:12.820 INFO:teuthology.orchestra.run.smithi103.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-11T00:09:12.821 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-11T00:09:12.871 INFO:teuthology.orchestra.run.smithi103.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-11T00:09:12.929 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-11T00:09:12.932 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-11T00:09:12.944 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:09:12.988 INFO:teuthology.orchestra.run.smithi103.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-11T00:09:13.066 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:09:13.071 INFO:teuthology.orchestra.run.smithi103.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-09-11T00:09:13.095 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:09:13.147 INFO:teuthology.orchestra.run.smithi103.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 544 kB/s | 41 kB 00:00 2024-09-11T00:09:13.170 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:09:13.197 INFO:teuthology.orchestra.run.smithi103.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-11T00:09:13.199 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:09:13.228 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-11T00:09:13.289 INFO:teuthology.orchestra.run.smithi103.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-11T00:09:13.347 INFO:teuthology.orchestra.run.smithi103.stdout:(155/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 6.2 MB/s | 3.2 MB 00:00 2024-09-11T00:09:13.383 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-11T00:09:13.406 INFO:teuthology.orchestra.run.smithi103.stdout:(156/157): librados2-19.3.0-4854.g8f219c47.el9. 5.6 MB/s | 3.5 MB 00:00 2024-09-11T00:09:13.430 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-11T00:09:13.456 INFO:teuthology.orchestra.run.smithi103.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-11T00:09:13.458 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:09:13.461 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:09:13.461 INFO:teuthology.orchestra.run.smithi103.stdout:Total 13 MB/s | 240 MB 00:18 2024-09-11T00:09:13.562 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:09:13.593 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:09:13.655 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:09:13.677 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:09:13.700 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:09:13.895 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:09:14.038 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:09:14.160 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-11T00:09:14.250 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:14.310 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:14.524 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:14.637 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-11T00:09:14.681 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-11T00:09:14.724 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-11T00:09:14.798 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-11T00:09:14.819 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-11T00:09:14.819 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-11T00:09:14.851 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-11T00:09:14.890 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-11T00:09:14.933 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-11T00:09:14.966 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-11T00:09:14.994 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-11T00:09:15.064 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-11T00:09:15.125 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-11T00:09:15.155 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-11T00:09:15.190 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-11T00:09:15.269 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-11T00:09:15.301 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-11T00:09:15.344 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-11T00:09:15.373 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-11T00:09:15.446 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-11T00:09:15.491 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-11T00:09:15.539 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-09-11T00:09:15.565 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-11T00:09:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-11T00:09:15.731 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-09-11T00:09:15.731 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-09-11T00:09:15.751 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-11T00:09:16.289 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-11T00:09:16.365 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-11T00:09:16.393 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-11T00:09:16.436 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-11T00:09:16.474 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-11T00:09:16.515 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-11T00:09:16.669 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-11T00:09:16.727 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-11T00:09:16.819 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-11T00:09:16.883 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-11T00:09:16.963 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-11T00:09:17.004 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-11T00:09:17.039 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-11T00:09:17.087 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-11T00:09:17.122 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-11T00:09:17.166 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-11T00:09:17.207 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-11T00:09:17.343 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-11T00:09:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-11T00:09:17.419 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-11T00:09:17.523 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-11T00:09:17.560 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-11T00:09:17.600 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-11T00:09:17.751 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-11T00:09:17.776 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-11T00:09:17.850 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-11T00:09:17.978 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-11T00:09:18.025 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-11T00:09:18.053 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-11T00:09:18.087 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-11T00:09:18.126 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-11T00:09:18.166 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-11T00:09:18.204 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-11T00:09:18.238 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-11T00:09:18.241 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-09-11T00:09:18.242 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-09-11T00:09:18.272 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-11T00:09:18.313 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-11T00:09:18.348 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-11T00:09:18.435 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-11T00:09:18.478 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-11T00:09:18.511 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-11T00:09:18.614 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-11T00:09:18.671 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-11T00:09:19.405 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-11T00:09:19.451 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-11T00:09:19.485 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-11T00:09:19.587 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-11T00:09:19.640 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-11T00:09:19.698 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-11T00:09:19.788 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-11T00:09:19.829 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-11T00:09:20.013 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-11T00:09:20.043 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-11T00:09:20.121 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-11T00:09:20.764 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-11T00:09:20.874 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-11T00:09:21.175 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-11T00:09:21.290 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-11T00:09:21.413 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-11T00:09:21.686 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-11T00:09:21.743 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-11T00:09:21.767 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-11T00:09:21.820 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-11T00:09:22.113 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-11T00:09:22.154 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-11T00:09:22.197 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-11T00:09:22.225 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-11T00:09:22.258 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-11T00:09:22.259 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:09:22.381 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:09:22.402 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:09:22.493 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:09:22.523 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:09:22.552 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-11T00:09:22.581 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-11T00:09:22.629 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:09:22.708 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-11T00:09:22.720 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:09:22.763 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-11T00:09:22.791 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:09:22.894 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:09:22.917 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:09:22.968 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-11T00:09:22.979 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:09:23.001 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:09:23.003 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-11T00:09:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:09:23.220 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:09:23.373 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:09:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-11T00:09:23.508 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-11T00:09:23.540 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:09:23.569 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:23.611 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:09:23.642 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-11T00:09:23.642 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:23.851 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-11T00:09:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-11T00:09:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-11T00:09:24.095 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-11T00:09:24.141 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-11T00:09:24.173 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-11T00:09:24.216 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-11T00:09:24.249 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-11T00:09:24.277 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-11T00:09:24.345 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-11T00:09:24.408 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-11T00:09:24.437 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-11T00:09:24.473 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-11T00:09:24.551 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-11T00:09:24.593 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-11T00:09:24.636 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-11T00:09:24.665 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-11T00:09:24.736 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-11T00:09:24.781 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-11T00:09:24.855 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-11T00:09:24.999 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-11T00:09:25.058 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-11T00:09:25.179 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-09-11T00:09:25.403 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-11T00:09:25.607 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-11T00:09:25.691 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-11T00:09:25.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-11T00:09:25.718 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-11T00:09:25.761 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-11T00:09:25.799 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-11T00:09:25.841 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-11T00:09:25.862 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-11T00:09:25.906 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-11T00:09:25.938 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-11T00:09:25.994 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-11T00:09:25.996 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:26.013 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-11T00:09:26.029 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:26.092 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-11T00:09:26.185 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-11T00:09:26.241 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-11T00:09:26.314 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-11T00:09:26.346 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-11T00:09:26.361 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-11T00:09:26.382 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-11T00:09:26.428 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-11T00:09:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-11T00:09:26.491 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:09:26.516 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-11T00:09:26.561 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-11T00:09:26.585 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:26.624 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:09:26.641 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:09:26.673 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:09:26.716 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:09:26.716 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T00:09:26.716 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T00:09:26.716 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:26.751 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:09:26.761 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:09:26.798 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:09:26.841 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-11T00:09:26.854 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-11T00:09:26.943 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-11T00:09:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-11T00:09:27.077 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-11T00:09:27.088 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-11T00:09:27.118 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-11T00:09:27.145 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-11T00:09:27.158 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-11T00:09:27.188 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:09:27.192 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-11T00:09:27.210 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-11T00:09:27.285 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-11T00:09:27.363 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:09:27.388 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:09:27.413 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-11T00:09:27.444 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-11T00:09:27.472 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-11T00:09:27.504 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-11T00:09:27.506 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-11T00:09:27.545 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-11T00:09:27.556 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:09:27.577 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-11T00:09:27.598 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-11T00:09:27.607 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-11T00:09:27.641 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-11T00:09:27.674 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-11T00:09:27.716 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-11T00:09:27.750 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-11T00:09:27.834 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:09:27.837 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-11T00:09:27.881 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-11T00:09:27.921 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-11T00:09:28.024 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-11T00:09:28.041 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-11T00:09:28.071 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-11T00:09:28.279 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-11T00:09:28.411 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:28.507 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:28.794 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-11T00:09:28.825 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-11T00:09:28.851 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-11T00:09:28.885 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-11T00:09:28.972 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-11T00:09:28.983 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-11T00:09:29.015 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-11T00:09:29.041 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-11T00:09:29.057 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-11T00:09:29.085 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-11T00:09:29.138 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-11T00:09:29.164 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-11T00:09:29.201 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-11T00:09:29.205 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-11T00:09:29.249 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-11T00:09:29.330 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-11T00:09:29.383 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-11T00:09:29.388 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-11T00:09:29.417 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-11T00:09:29.419 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-11T00:09:29.496 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-11T00:09:29.505 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-11T00:09:29.569 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-11T00:09:29.605 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-11T00:09:29.677 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-11T00:09:29.765 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-11T00:09:29.808 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-11T00:09:29.852 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-11T00:09:29.881 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-11T00:09:29.970 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-11T00:09:30.041 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-11T00:09:30.130 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-11T00:09:30.132 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-11T00:09:30.242 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-11T00:09:30.290 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-11T00:09:30.379 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-11T00:09:31.009 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-11T00:09:31.120 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-11T00:09:31.157 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-11T00:09:31.202 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-11T00:09:31.239 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-11T00:09:31.295 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-11T00:09:31.469 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-11T00:09:31.477 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-11T00:09:31.534 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-11T00:09:31.556 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-11T00:09:31.584 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-11T00:09:31.617 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-11T00:09:31.647 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-11T00:09:31.711 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-11T00:09:31.813 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-11T00:09:31.853 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-11T00:09:31.889 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-11T00:09:31.937 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-11T00:09:31.964 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-11T00:09:31.980 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:09:32.005 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-11T00:09:32.050 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-11T00:09:32.097 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:09:32.126 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-11T00:09:32.127 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-11T00:09:32.163 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-11T00:09:32.297 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-11T00:09:32.346 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-11T00:09:32.389 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-11T00:09:32.561 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-11T00:09:32.614 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-11T00:09:32.655 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-11T00:09:32.656 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-11T00:09:32.683 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:09:32.686 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-11T00:09:32.711 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-11T00:09:32.754 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:09:32.776 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-11T00:09:32.862 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-11T00:09:33.029 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-11T00:09:33.077 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-11T00:09:33.139 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-11T00:09:33.177 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-11T00:09:33.222 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-11T00:09:33.262 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-11T00:09:33.299 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-11T00:09:33.334 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-11T00:09:33.367 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-11T00:09:33.410 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-11T00:09:33.462 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-11T00:09:33.901 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-11T00:09:33.952 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-11T00:09:33.989 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-11T00:09:34.109 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-11T00:09:34.205 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-11T00:09:35.055 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:35.089 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-11T00:09:35.090 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:35.153 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-11T00:09:35.187 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-11T00:09:35.316 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-11T00:09:35.387 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-11T00:09:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-11T00:09:35.535 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-11T00:09:35.581 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-11T00:09:35.646 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:35.685 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:09:35.776 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:09:35.776 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T00:09:35.776 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T00:09:35.776 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:35.803 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:09:35.816 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-11T00:09:35.853 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-11T00:09:35.989 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-11T00:09:36.815 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-11T00:09:36.958 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-11T00:09:38.311 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /root 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2024-09-11T00:09:38.312 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:38.487 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:09:38.504 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:09:38.504 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:38.504 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T00:09:38.504 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:38.512 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-11T00:09:38.659 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-11T00:09:38.696 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-11T00:09:38.729 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-11T00:09:39.075 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-11T00:09:39.182 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:09:39.327 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-11T00:09:39.372 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-11T00:09:39.420 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-11T00:09:39.958 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-11T00:09:39.986 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:09:39.998 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:09:40.025 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:09:40.025 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:40.025 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T00:09:40.026 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:09:40.026 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:09:40.026 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:40.085 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-11T00:09:40.113 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-11T00:09:40.249 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:09:40.273 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:09:40.273 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:40.273 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T00:09:40.273 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:09:40.273 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:09:40.273 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:40.724 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:09:40.748 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:09:40.748 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:40.748 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T00:09:40.748 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:09:40.749 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:09:40.749 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:40.855 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-11T00:09:40.912 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-11T00:09:40.967 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-11T00:09:41.153 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-11T00:09:41.201 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-11T00:09:41.278 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-11T00:09:41.367 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-11T00:09:41.506 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-11T00:09:41.542 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-11T00:09:41.574 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-11T00:09:41.602 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-11T00:09:42.073 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:09:42.100 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:09:42.132 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-11T00:09:42.191 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-11T00:09:42.215 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:09:42.800 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:42.851 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:43.497 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-11T00:09:43.547 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:09:43.671 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-11T00:09:43.671 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T00:09:43.671 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T00:09:43.671 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:09:43.723 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:09:45.395 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:09:45.454 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:09:46.415 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:09:46.446 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:09:46.559 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:09:46.716 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-11T00:09:46.756 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:09:46.780 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:09:46.780 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:46.780 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T00:09:46.781 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:09:46.781 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:09:46.781 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:46.832 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:09:46.843 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:09:46.941 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-11T00:09:46.988 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-11T00:09:47.380 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:09:47.380 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-09-11T00:09:47.380 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-09-11T00:09:47.381 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-09-11T00:09:47.381 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-09-11T00:09:47.381 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-09-11T00:09:47.381 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-09-11T00:09:47.381 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-09-11T00:09:47.381 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:47.548 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:09:47.563 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:09:47.564 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:47.564 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T00:09:47.564 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:47.933 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-11T00:09:47.959 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:09:47.983 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:09:47.983 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:47.983 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T00:09:47.983 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:09:47.983 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:09:47.983 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:48.026 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:09:48.049 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:09:48.050 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:48.050 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T00:09:48.050 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:48.102 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:09:48.126 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:09:48.126 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:48.126 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T00:09:48.126 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:09:48.126 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:09:48.126 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:48.436 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:09:48.459 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:09:48.460 INFO:teuthology.orchestra.run.smithi117.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:48.460 INFO:teuthology.orchestra.run.smithi117.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T00:09:48.460 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:09:48.460 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:09:48.460 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:09:49.067 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:09:49.093 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:09:49.093 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:49.094 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T00:09:49.094 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:09:49.094 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:09:49.094 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:49.318 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:09:49.341 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:09:49.341 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:49.341 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T00:09:49.341 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:09:49.341 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:09:49.341 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:49.785 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:09:49.808 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:09:49.808 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:49.808 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T00:09:49.808 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:09:49.808 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:09:49.809 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:49.907 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-11T00:09:49.964 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-11T00:09:50.028 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-11T00:09:50.216 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-11T00:09:50.261 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-11T00:09:50.331 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-11T00:09:50.421 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-11T00:09:50.559 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-11T00:09:50.594 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-11T00:09:50.626 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-11T00:09:50.654 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-11T00:09:51.034 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:09:51.070 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:09:51.101 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-11T00:09:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-11T00:09:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:09:53.131 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-11T00:09:53.172 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-11T00:09:53.213 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-11T00:09:53.245 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-11T00:09:53.273 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-11T00:09:53.455 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-11T00:09:53.495 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-11T00:09:53.525 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-11T00:09:53.554 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-11T00:09:53.600 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-11T00:09:54.220 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:09:54.257 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:09:54.277 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:09:54.358 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:09:54.415 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:09:55.096 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-11T00:09:55.195 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-11T00:09:55.235 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-11T00:09:55.316 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-11T00:09:55.348 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-11T00:09:55.373 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-11T00:09:55.377 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:09:55.399 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-11T00:09:55.399 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:09:55.407 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:09:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:09:55.483 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:09:55.520 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:09:55.567 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:09:55.592 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-11T00:09:55.597 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:09:55.598 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:09:55.678 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-11T00:09:55.709 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:09:55.732 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:09:55.732 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:55.732 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T00:09:55.733 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:09:55.733 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:09:55.733 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:55.776 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:09:55.785 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:09:55.785 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:09:55.788 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:09:55.870 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:09:55.889 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-11T00:09:55.932 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-11T00:09:55.933 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:09:56.715 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-11T00:09:56.715 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-09-11T00:09:56.715 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-09-11T00:09:56.715 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-09-11T00:09:56.716 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-09-11T00:09:56.716 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-09-11T00:09:56.716 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-09-11T00:09:56.716 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-09-11T00:09:56.716 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:09:56.885 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-11T00:09:56.924 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:09:56.929 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:09:56.952 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:09:56.952 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:56.952 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T00:09:56.952 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:09:56.952 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:09:56.953 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:56.966 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-11T00:09:56.966 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:56.967 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T00:09:56.967 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:09:56.996 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:09:57.019 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:09:57.019 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:57.019 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T00:09:57.019 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:57.064 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:09:57.086 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:09:57.087 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:57.087 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T00:09:57.087 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:09:57.087 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:09:57.087 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:57.398 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:09:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:09:57.421 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:57.421 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T00:09:57.421 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:09:57.421 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:09:57.421 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:09:58.584 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:09:58.642 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-11T00:09:58.642 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:58.643 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T00:09:58.643 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:09:58.643 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T00:09:58.643 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:09:58.893 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:09:58.945 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-11T00:09:58.945 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:58.945 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T00:09:58.945 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:09:58.945 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T00:09:58.945 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:09:59.436 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:09:59.488 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-11T00:09:59.488 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:09:59.488 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T00:09:59.488 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:09:59.488 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T00:09:59.488 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:09:59.620 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-11T00:09:59.688 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-11T00:09:59.786 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-11T00:10:00.010 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-11T00:10:00.098 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-11T00:10:00.184 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-11T00:10:00.302 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-11T00:10:00.469 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-11T00:10:00.533 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-11T00:10:00.572 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-11T00:10:00.598 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-11T00:10:01.376 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:10:01.407 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-11T00:10:01.433 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-11T00:10:01.522 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-11T00:10:01.563 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:10:02.093 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-11T00:10:02.134 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-11T00:10:02.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-11T00:10:02.207 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-11T00:10:02.237 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-11T00:10:02.325 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:02.325 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-11T00:10:02.325 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-11T00:10:02.325 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-11T00:10:02.325 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-11T00:10:02.325 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-11T00:10:02.326 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-11T00:10:02.327 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-11T00:10:02.328 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-11T00:10:02.339 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-11T00:10:02.339 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-11T00:10:02.339 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-11T00:10:02.339 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-11T00:10:02.339 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-11T00:10:02.339 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-11T00:10:02.340 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-11T00:10:02.341 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-11T00:10:02.342 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-11T00:10:02.343 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-11T00:10:02.344 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-11T00:10:02.345 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-11T00:10:02.346 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-11T00:10:02.347 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-11T00:10:02.348 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-11T00:10:02.349 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-11T00:10:02.416 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-11T00:10:02.456 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-11T00:10:02.478 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-11T00:10:02.507 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-11T00:10:02.554 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-11T00:10:03.126 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:10:03.168 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:10:03.189 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:10:04.021 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-11T00:10:04.107 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-11T00:10:04.148 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-11T00:10:04.229 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-11T00:10:04.268 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-11T00:10:04.293 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-11T00:10:04.319 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-11T00:10:04.320 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:10:04.395 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:10:04.395 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:10:04.462 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:10:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-11T00:10:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:10:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:10:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:10:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:10:04.757 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:10:04.819 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:04.886 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:04.886 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:10:04.886 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2024-09-11T00:10:04.886 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:04.886 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:04.886 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T00:10:04.887 INFO:teuthology.orchestra.run.smithi117.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.888 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.889 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T00:10:04.890 INFO:teuthology.orchestra.run.smithi117.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.891 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T00:10:04.892 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.893 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T00:10:04.894 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T00:10:04.895 INFO:teuthology.orchestra.run.smithi117.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.896 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T00:10:04.897 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.898 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:10:04.899 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-11T00:10:05.175 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-11T00:10:05.261 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:10:05.332 DEBUG:teuthology.parallel:result is None 2024-09-11T00:10:06.348 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-11T00:10:06.394 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:10:06.545 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-11T00:10:06.725 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-11T00:10:06.780 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:10:06.832 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-11T00:10:06.832 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:10:06.832 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T00:10:06.833 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:10:06.833 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T00:10:06.833 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:06.890 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:10:06.915 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-11T00:10:07.023 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-11T00:10:07.105 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-11T00:10:08.139 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-11T00:10:08.175 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:10:08.227 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-11T00:10:08.227 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:10:08.227 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T00:10:08.228 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:10:08.228 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T00:10:08.228 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:08.276 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:10:08.326 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-11T00:10:08.394 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:10:08.394 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T00:10:08.394 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:08.395 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:10:08.427 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-11T00:10:08.427 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:10:08.427 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T00:10:08.428 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:10:08.428 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T00:10:08.428 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:08.794 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:10:08.845 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-11T00:10:08.845 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T00:10:08.845 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T00:10:08.845 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:10:08.845 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T00:10:08.845 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:11.063 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:11.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-11T00:10:11.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-11T00:10:11.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-11T00:10:11.063 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:10:11.064 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-11T00:10:11.065 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-11T00:10:11.066 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-11T00:10:11.067 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-11T00:10:11.068 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-11T00:10:11.069 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-11T00:10:11.070 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-11T00:10:11.071 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-11T00:10:11.072 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-11T00:10:11.073 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-11T00:10:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-11T00:10:11.075 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.596 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.597 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.598 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T00:10:13.599 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T00:10:13.600 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T00:10:13.601 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T00:10:13.602 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T00:10:13.603 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T00:10:13.604 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T00:10:13.605 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T00:10:13.606 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T00:10:13.607 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T00:10:13.608 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T00:10:13.609 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T00:10:13.609 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:10:13.609 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-11T00:10:13.965 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-11T00:10:14.016 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-11T00:10:14.029 DEBUG:teuthology.parallel:result is None 2024-09-11T00:10:14.060 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-11T00:10:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-11T00:10:14.125 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-11T00:10:14.363 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-11T00:10:14.414 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-11T00:10:14.445 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-11T00:10:14.484 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-11T00:10:14.519 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-11T00:10:15.369 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:10:15.418 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:10:15.459 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-11T00:10:16.727 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-11T00:10:16.833 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-11T00:10:16.890 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-11T00:10:16.995 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-11T00:10:17.036 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-11T00:10:17.093 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-11T00:10:17.131 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-11T00:10:17.131 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:10:17.239 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-11T00:10:17.239 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:10:17.348 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-11T00:10:17.372 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-11T00:10:17.387 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:10:17.387 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:10:17.633 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-11T00:10:17.633 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:10:17.743 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-11T00:10:17.840 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:18.053 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:18.054 INFO:teuthology.orchestra.run.smithi103.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-11T00:10:18.054 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:24.625 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-11T00:10:24.625 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-11T00:10:24.625 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-11T00:10:24.625 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-11T00:10:24.625 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-11T00:10:24.626 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-11T00:10:24.627 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-11T00:10:24.639 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-11T00:10:24.639 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-11T00:10:24.639 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-11T00:10:24.640 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-11T00:10:24.641 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-11T00:10:24.642 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-11T00:10:24.643 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-11T00:10:24.644 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-11T00:10:24.645 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-11T00:10:24.646 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-11T00:10:24.647 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-11T00:10:24.648 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-11T00:10:24.649 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-11T00:10:24.650 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-11T00:10:24.650 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-11T00:10:24.650 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-11T00:10:26.952 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-11T00:10:26.952 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:26.952 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-09-11T00:10:26.952 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:26.952 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:26.952 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:26.952 INFO:teuthology.orchestra.run.smithi103.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.953 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.954 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T00:10:26.955 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T00:10:26.956 INFO:teuthology.orchestra.run.smithi103.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T00:10:26.957 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T00:10:26.958 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T00:10:26.959 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T00:10:26.960 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T00:10:26.961 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.962 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T00:10:26.963 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T00:10:26.964 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T00:10:26.965 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T00:10:26.965 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T00:10:26.965 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:10:26.965 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-09-11T00:10:27.633 DEBUG:teuthology.parallel:result is None 2024-09-11T00:10:27.634 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:10:27.790 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T00:10:27.837 INFO:teuthology.orchestra.run.smithi103.stdout:19.3.0-4854.g8f219c47.el9 2024-09-11T00:10:27.838 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-11T00:10:27.838 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-11T00:10:27.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:10:27.995 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T00:10:28.021 INFO:teuthology.orchestra.run.smithi117.stdout:19.3.0-4854.g8f219c47.el9 2024-09-11T00:10:28.022 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-11T00:10:28.022 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-11T00:10:28.024 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:10:28.170 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T00:10:28.197 INFO:teuthology.orchestra.run.smithi183.stdout:19.3.0-4854.g8f219c47.el9 2024-09-11T00:10:28.197 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-11T00:10:28.197 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-11T00:10:28.199 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T00:10:28.199 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:10:28.199 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T00:10:28.263 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:10:28.263 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T00:10:28.296 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:10:28.296 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T00:10:28.328 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T00:10:28.328 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:10:28.328 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T00:10:28.386 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T00:10:28.486 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:10:28.486 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T00:10:28.516 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T00:10:28.585 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:10:28.585 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T00:10:28.615 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T00:10:28.683 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T00:10:28.683 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:10:28.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T00:10:28.745 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T00:10:28.842 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:10:28.843 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T00:10:28.868 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T00:10:28.933 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:10:28.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T00:10:28.957 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T00:10:29.024 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T00:10:29.024 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:10:29.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T00:10:29.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T00:10:29.183 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:10:29.183 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T00:10:29.209 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T00:10:29.274 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:10:29.274 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T00:10:29.300 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T00:10:29.364 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T00:10:29.511 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-11T00:10:29.512 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:10:29.512 INFO:tasks.cephadm:Cluster fsid is 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:10:29.512 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T00:10:29.513 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.103', 'mon.b': '172.21.15.117', 'mon.c': '172.21.15.183'} 2024-09-11T00:10:29.513 INFO:tasks.cephadm:First mon is mon.a on smithi103 2024-09-11T00:10:29.513 INFO:tasks.cephadm:First mgr is x 2024-09-11T00:10:29.513 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T00:10:29.513 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-09-11T00:10:29.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-09-11T00:10:29.604 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-09-11T00:10:29.628 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T00:10:29.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:10:29.781 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-11T00:10:29.929 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:10:29.930 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-11T00:10:29.930 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-11T00:10:29.930 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:30.387 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 11 00:10 /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:30.387 DEBUG:teuthology.orchestra.run.smithi117:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:30.782 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 11 00:10 /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:30.783 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:31.162 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 11 00:10 /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:31.162 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:31.203 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:31.223 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T00:10:31.339 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-11T00:10:31.339 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-11T00:10:31.344 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-11T00:10:31.347 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-11T00:10:32.798 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:10:32.931 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:10:33.445 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:11:03.200 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-11T00:11:03.200 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:11:03.200 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-11T00:11:03.200 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-09-11T00:11:03.200 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-11T00:11:03.201 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-11T00:11:03.201 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-11T00:11:04.920 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-11T00:11:04.920 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:11:04.920 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-11T00:11:04.920 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-09-11T00:11:04.920 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-11T00:11:04.920 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-11T00:11:04.920 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-11T00:11:08.221 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:11:08.221 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:11:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-11T00:11:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-09-11T00:11:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-11T00:11:08.222 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:11:08.222 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:11:08.253 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-09-11T00:11:08.313 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-09-11T00:11:08.347 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-09-11T00:11:08.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-09-11T00:11:08.435 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-09-11T00:11:08.465 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-09-11T00:11:08.493 INFO:tasks.cephadm:Writing seed config... 2024-09-11T00:11:08.494 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T00:11:08.494 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T00:11:08.495 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T00:11:08.496 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T00:11:08.497 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:11:08.497 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T00:11:08.531 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 40f10072-6fd2-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-09-11T00:11:08.532 DEBUG:teuthology.orchestra.run.smithi103:mon.a> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T00:11:08.576 DEBUG:teuthology.orchestra.run.smithi103:mgr.x> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T00:11:08.619 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T00:11:08.620 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.103 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:11:08.883 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-09-11T00:11:08.883 INFO:teuthology.orchestra.run.smithi103.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.103', '--skip-admin-label'] 2024-09-11T00:11:08.883 INFO:teuthology.orchestra.run.smithi103.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T00:11:08.884 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying podman|docker is present... 2024-09-11T00:11:08.927 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-11T00:11:08.927 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying lvm2 is present... 2024-09-11T00:11:08.927 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying time synchronization is in place... 2024-09-11T00:11:08.941 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T00:11:08.941 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T00:11:08.953 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T00:11:08.953 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-11T00:11:08.967 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-11T00:11:08.980 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-11T00:11:08.980 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-11T00:11:08.980 INFO:teuthology.orchestra.run.smithi103.stdout:Repeating the final host check... 2024-09-11T00:11:09.023 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 5.2.2 2024-09-11T00:11:09.023 INFO:teuthology.orchestra.run.smithi103.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T00:11:09.023 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl is present 2024-09-11T00:11:09.023 INFO:teuthology.orchestra.run.smithi103.stdout:lvcreate is present 2024-09-11T00:11:09.036 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T00:11:09.036 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T00:11:09.048 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T00:11:09.048 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-11T00:11:09.061 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout enabled 2024-09-11T00:11:09.074 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout active 2024-09-11T00:11:09.074 INFO:teuthology.orchestra.run.smithi103.stdout:Unit chronyd.service is enabled and running 2024-09-11T00:11:09.074 INFO:teuthology.orchestra.run.smithi103.stdout:Host looks OK 2024-09-11T00:11:09.074 INFO:teuthology.orchestra.run.smithi103.stdout:Cluster fsid: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:09.074 INFO:teuthology.orchestra.run.smithi103.stdout:Acquiring lock 139747801469808 on /run/cephadm/40f10072-6fd2-11ef-bcea-c7b262605968.lock 2024-09-11T00:11:09.075 INFO:teuthology.orchestra.run.smithi103.stdout:Lock 139747801469808 acquired on /run/cephadm/40f10072-6fd2-11ef-bcea-c7b262605968.lock 2024-09-11T00:11:09.075 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 3300 ... 2024-09-11T00:11:09.076 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying IP 172.21.15.103 port 6789 ... 2024-09-11T00:11:09.076 INFO:teuthology.orchestra.run.smithi103.stdout:Base mon IP(s) is [172.21.15.103:3300, 172.21.15.103:6789], mon addrv is [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-11T00:11:09.083 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.103 metric 100 2024-09-11T00:11:09.083 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.103 metric 100 2024-09-11T00:11:09.088 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T00:11:09.089 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T00:11:09.089 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T00:11:09.094 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T00:11:09.094 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T00:11:09.094 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T00:11:09.094 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T00:11:09.095 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-11T00:11:09.095 INFO:teuthology.orchestra.run.smithi103.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T00:11:09.095 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-11T00:11:09.095 INFO:teuthology.orchestra.run.smithi103.stdout:Mon IP `172.21.15.103` is in CIDR network `172.21.0.0/20` 2024-09-11T00:11:09.095 INFO:teuthology.orchestra.run.smithi103.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T00:11:09.096 INFO:teuthology.orchestra.run.smithi103.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T00:11:09.097 INFO:teuthology.orchestra.run.smithi103.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:11:09.643 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-11T00:11:09.643 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-11T00:11:09.643 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T00:11:09.643 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-11T00:11:09.643 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T00:11:09.643 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-11T00:11:09.643 INFO:teuthology.orchestra.run.smithi103.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T00:11:21.912 INFO:teuthology.orchestra.run.smithi103.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-11T00:11:21.912 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-11T00:11:21.912 INFO:teuthology.orchestra.run.smithi103.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T00:11:23.114 INFO:teuthology.orchestra.run.smithi103.stdout:stat: stdout 167 167 2024-09-11T00:11:23.114 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial keys... 2024-09-11T00:11:24.127 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAr4OBmBaErHRAAeI4qB2adzgIHRGWrKVI+rQ== 2024-09-11T00:11:25.275 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAs4OBmj5ReHhAA7s32DjWhN40+rC+IxW1N3w== 2024-09-11T00:11:26.279 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph-authtool: stdout AQAt4OBmrKgcJhAAeBOOFSPLFkngheqEdI4kaQ== 2024-09-11T00:11:26.280 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial monmap... 2024-09-11T00:11:27.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T00:11:27.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T00:11:27.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:27.678 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T00:11:27.678 INFO:teuthology.orchestra.run.smithi103.stdout:monmaptool for a [v2:172.21.15.103:3300,v1:172.21.15.103:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T00:11:27.678 INFO:teuthology.orchestra.run.smithi103.stdout:setting min_mon_release = quincy 2024-09-11T00:11:27.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: set fsid to 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:27.679 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T00:11:27.679 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:11:27.679 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mon... 2024-09-11T00:11:28.836 INFO:teuthology.orchestra.run.smithi103.stdout:create mon.a on 2024-09-11T00:11:29.148 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T00:11:29.367 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T00:11:29.660 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-40f10072-6fd2-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-40f10072-6fd2-11ef-bcea-c7b262605968.target. 2024-09-11T00:11:29.660 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-40f10072-6fd2-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-40f10072-6fd2-11ef-bcea-c7b262605968.target. 2024-09-11T00:11:30.015 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a 2024-09-11T00:11:30.015 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T00:11:30.297 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-40f10072-6fd2-11ef-bcea-c7b262605968.target.wants/ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-40f10072-6fd2-11ef-bcea-c7b262605968@.service. 2024-09-11T00:11:30.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:30 smithi103 podman[30191]: 2024-09-11 00:11:30.569966805 +0000 UTC m=+0.027051847 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:11:31.051 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:11:31.052 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-11T00:11:31.065 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:11:31.065 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-11T00:11:31.065 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-11T00:11:31.065 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T00:11:31.065 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon to start... 2024-09-11T00:11:31.065 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mon... 2024-09-11T00:11:31.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:30 smithi103 podman[30191]: 2024-09-11 00:11:30.93261902 +0000 UTC m=+0.389704062 container init 7fe3feae8261b9507d495eff8ed76ac814e2880d4c49398014e36f5f5f0dd065 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-11T00:11:31.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:30 smithi103 podman[30191]: 2024-09-11 00:11:30.939073518 +0000 UTC m=+0.396158567 container start 7fe3feae8261b9507d495eff8ed76ac814e2880d4c49398014e36f5f5f0dd065 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T00:11:31.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:31 smithi103 ceph-mon[30210]: mkfs 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:31.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:31 smithi103 ceph-mon[30210]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T00:11:31.206 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:31 smithi103 bash[30191]: 7fe3feae8261b9507d495eff8ed76ac814e2880d4c49398014e36f5f5f0dd065 2024-09-11T00:11:31.207 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:31 smithi103 systemd[1]: Started Ceph mon.a for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:11:32.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: monmap epoch 1 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: last_changed 2024-09-11T00:11:26.844308+0000 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: created 2024-09-11T00:11:26.844308+0000 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: min_mon_release 19 (squid) 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: election_strategy: 1 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: fsmap 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: mgrmap e1: no daemons active 2024-09-11T00:11:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:32 smithi103 ceph-mon[30210]: from='client.? 172.21.15.103:0/323957264' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-11T00:11:33.005 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout id: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout services: 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.659125s) 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout data: 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T00:11:33.007 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T00:11:33.008 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T00:11:33.008 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T00:11:33.008 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:33.008 INFO:teuthology.orchestra.run.smithi103.stdout:mon is available 2024-09-11T00:11:33.008 INFO:teuthology.orchestra.run.smithi103.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T00:11:33.863 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:33 smithi103 ceph-mon[30210]: from='client.? 172.21.15.103:0/2603497562' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-11T00:11:33.863 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:33 smithi103 ceph-mon[30210]: from='client.? 172.21.15.103:0/2603497562' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T00:11:34.178 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T00:11:34.179 INFO:teuthology.orchestra.run.smithi103.stdout:Generating new minimal ceph.conf... 2024-09-11T00:11:35.090 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:34 smithi103 ceph-mon[30210]: from='client.? 172.21.15.103:0/1729404507' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:11:35.384 INFO:teuthology.orchestra.run.smithi103.stdout:Restarting the monitor... 2024-09-11T00:11:35.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:35 smithi103 systemd[1]: Stopping Ceph mon.a for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:11:36.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:35 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a[30206]: 2024-09-11T00:11:35.781+0000 7f4e74a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:11:36.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:35 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a[30206]: 2024-09-11T00:11:35.781+0000 7f4e74a00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T00:11:36.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:35 smithi103 podman[30448]: 2024-09-11 00:11:35.967683871 +0000 UTC m=+0.348907030 container died 7fe3feae8261b9507d495eff8ed76ac814e2880d4c49398014e36f5f5f0dd065 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:11:36.777 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:36 smithi103 podman[30448]: 2024-09-11 00:11:36.436823089 +0000 UTC m=+0.818046257 container cleanup 7fe3feae8261b9507d495eff8ed76ac814e2880d4c49398014e36f5f5f0dd065 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T00:11:36.777 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:36 smithi103 bash[30448]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a 2024-09-11T00:11:36.777 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:36 smithi103 podman[30460]: 2024-09-11 00:11:36.646098617 +0000 UTC m=+0.659642977 container remove 7fe3feae8261b9507d495eff8ed76ac814e2880d4c49398014e36f5f5f0dd065 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:11:37.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:36 smithi103 systemd[1]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T00:11:37.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:36 smithi103 systemd[1]: Stopped Ceph mon.a for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:11:37.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:36 smithi103 systemd[1]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.253s CPU time. 2024-09-11T00:11:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:36 smithi103 systemd[1]: Starting Ceph mon.a for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:11:37.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 podman[30559]: 2024-09-11 00:11:37.173807887 +0000 UTC m=+0.027748802 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:11:37.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 podman[30559]: 2024-09-11 00:11:37.282364233 +0000 UTC m=+0.136305148 container create 0cabdefb1d1cf21f3b86fee4007caef54c51dd9f0be7da85def4f7a0745cef9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:11:37.986 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 podman[30559]: 2024-09-11 00:11:37.893756047 +0000 UTC m=+0.747696961 container init 0cabdefb1d1cf21f3b86fee4007caef54c51dd9f0be7da85def4f7a0745cef9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T00:11:37.986 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 podman[30559]: 2024-09-11 00:11:37.900849263 +0000 UTC m=+0.754790179 container start 0cabdefb1d1cf21f3b86fee4007caef54c51dd9f0be7da85def4f7a0745cef9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:11:37.986 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T00:11:37.986 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: pidfile_write: ignore empty --pid-file 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: load: jerasure load: lrc 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: RocksDB version: 7.9.2 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Git sha 0 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: DB SUMMARY 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: DB Session ID: 49FVROZ7R0WVT0BO5TOB 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: CURRENT file: CURRENT 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95174 ; 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.error_if_exists: 0 2024-09-11T00:11:37.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.create_if_missing: 0 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.paranoid_checks: 1 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.env: 0x563bf3818180 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.info_log: 0x563bfd1716e0 2024-09-11T00:11:37.988 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T00:11:37.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.statistics: (nil) 2024-09-11T00:11:37.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.use_fsync: 0 2024-09-11T00:11:37.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_log_file_size: 0 2024-09-11T00:11:37.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T00:11:37.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T00:11:37.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.allow_fallocate: 1 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.use_direct_reads: 0 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.db_log_dir: 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.wal_dir: 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T00:11:37.990 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.write_buffer_manager: 0x563bfd175b80 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T00:11:37.991 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.unordered_write: 0 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.row_cache: None 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.wal_filter: None 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T00:11:37.992 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.two_write_queues: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.wal_compression: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.atomic_flush: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.log_readahead_size: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T00:11:37.993 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_background_jobs: 2 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_background_compactions: -1 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_subcompactions: 1 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T00:11:37.994 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_open_files: -1 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_background_flushes: -1 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Compression algorithms supported: 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kZSTD supported: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kXpressCompression supported: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kBZip2Compression supported: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kLZ4Compression supported: 1 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kZlibCompression supported: 1 2024-09-11T00:11:37.995 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: kSnappyCompression supported: 1 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.merge_operator: 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_filter: None 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T00:11:37.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563bfd171300) 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks: 1 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: pin_top_level_index_and_filter: 1 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_type: 0 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_index_type: 0 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_shortening: 1 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: checksum: 4 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: no_block_cache: 0 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache: 0x563bfd165610 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_name: BinnedLRUCache 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_options: 2024-09-11T00:11:37.997 INFO:journalctl@ceph.mon.a.smithi103.stdout: capacity : 536870912 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_shard_bits : 4 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: strict_capacity_limit : 0 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: high_pri_pool_ratio: 0.000 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_cache_compressed: (nil) 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: persistent_cache: (nil) 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size: 4096 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_size_deviation: 10 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_restart_interval: 16 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: index_block_restart_interval: 1 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: metadata_block_size: 4096 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: partition_filters: 0 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: use_delta_encoding: 1 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: filter_policy: bloomfilter 2024-09-11T00:11:37.998 INFO:journalctl@ceph.mon.a.smithi103.stdout: whole_key_filtering: 1 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: verify_compression: 0 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: read_amp_bytes_per_bit: 0 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: format_version: 5 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_index_compression: 1 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: block_align: 0 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: max_auto_readahead_size: 262144 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: prepopulate_block_cache: 0 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: initial_auto_readahead_size: 8192 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression: NoCompression 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T00:11:37.999 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.num_levels: 7 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T00:11:38.000 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T00:11:38.001 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T00:11:38.002 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T00:11:38.002 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T00:11:38.002 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T00:11:38.002 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T00:11:38.002 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T00:11:38.003 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.inplace_update_support: 0 2024-09-11T00:11:38.004 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.bloom_locality: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.max_successive_merges: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.ttl: 2592000 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T00:11:38.005 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.enable_blob_files: false 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.min_blob_size: 0 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 326974a7-9039-45b5-9581-aea979d60ca2 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726013497943200, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T00:11:38.006 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726013497944961, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91326, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 285, "table_properties": {"data_size": 89374, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11688, "raw_average_key_size": 47, "raw_value_size": 82793, "raw_average_value_size": 339, "num_data_blocks": 12, "num_entries": 244, "num_filter_entries": 244, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726013497, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "326974a7-9039-45b5-9581-aea979d60ca2", "db_session_id": "49FVROZ7R0WVT0BO5TOB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726013497945190, "job": 1, "event": "recovery_finished"} 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563bfd196e00 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: DB pointer 0x563bfd296000 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T00:11:38.007 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** DB Stats ** 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T00:11:38.008 INFO:journalctl@ceph.mon.a.smithi103.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: L0 2/0 91.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: Sum 2/0 91.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** Compaction Stats [default] ** 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 72.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-11T00:11:38.009 INFO:journalctl@ceph.mon.a.smithi103.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: Cumulative compaction: 0.00 GB write, 15.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: Interval compaction: 0.00 GB write, 15.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache BinnedLRUCache@0x563bfd165610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-11T00:11:38.010 INFO:journalctl@ceph.mon.a.smithi103.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: starting mon.a rank 0 at public addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] at bind addrs [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???) e1 preinit fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???).mds e1 new map 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???).mds e1 print_map 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout: e1 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout: btime 2024-09-11T00:11:30:997230+0000 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout: legacy client fscid: -1 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout: 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout: No filesystems configured 2024-09-11T00:11:38.011 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T00:11:38.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T00:11:38.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T00:11:38.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T00:11:38.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 ceph-mon[30578]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T00:11:38.012 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 bash[30559]: 0cabdefb1d1cf21f3b86fee4007caef54c51dd9f0be7da85def4f7a0745cef9e 2024-09-11T00:11:38.012 INFO:teuthology.orchestra.run.smithi103.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T00:11:38.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:37 smithi103 systemd[1]: Started Ceph mon.a for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:11:38.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T00:11:38.302 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: monmap epoch 1 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: last_changed 2024-09-11T00:11:26.844308+0000 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: created 2024-09-11T00:11:26.844308+0000 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: min_mon_release 19 (squid) 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: election_strategy: 1 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: fsmap 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T00:11:38.304 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:38 smithi103 ceph-mon[30578]: mgrmap e1: no daemons active 2024-09-11T00:11:39.274 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T00:11:39.300 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:11:39.300 INFO:teuthology.orchestra.run.smithi103.stdout:Creating mgr... 2024-09-11T00:11:39.301 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T00:11:39.301 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T00:11:39.649 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.x 2024-09-11T00:11:39.649 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Failed to reset failed state of unit ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T00:11:39.889 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-40f10072-6fd2-11ef-bcea-c7b262605968.target.wants/ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-40f10072-6fd2-11ef-bcea-c7b262605968@.service. 2024-09-11T00:11:39.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:39 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2313007010' entity='client.admin' 2024-09-11T00:11:40.235 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:39 smithi103 systemd[1]: Starting Ceph mgr.x for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:11:40.530 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:40 smithi103 podman[30786]: 2024-09-11 00:11:40.137947884 +0000 UTC m=+0.027435742 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:11:40.588 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:40 smithi103 podman[30786]: 2024-09-11 00:11:40.268691695 +0000 UTC m=+0.158179540 container create 5762ba2c7558eb5228a1a46d46dc6d101325ce9cba8622469fb0fb9729e9d2e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T00:11:40.589 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:40 smithi103 podman[30786]: 2024-09-11 00:11:40.44617974 +0000 UTC m=+0.335667586 container init 5762ba2c7558eb5228a1a46d46dc6d101325ce9cba8622469fb0fb9729e9d2e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T00:11:40.589 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:40 smithi103 podman[30786]: 2024-09-11 00:11:40.452159692 +0000 UTC m=+0.341647537 container start 5762ba2c7558eb5228a1a46d46dc6d101325ce9cba8622469fb0fb9729e9d2e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T00:11:40.589 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:40 smithi103 bash[30786]: 5762ba2c7558eb5228a1a46d46dc6d101325ce9cba8622469fb0fb9729e9d2e3 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:11:40.590 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-11T00:11:40.603 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:11:40.603 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-11T00:11:40.603 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-11T00:11:40.603 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T00:11:40.603 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr to start... 2024-09-11T00:11:40.603 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr... 2024-09-11T00:11:40.858 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:40 smithi103 systemd[1]: Started Ceph mgr.x for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:11:41.587 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:41 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/106736829' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-11T00:11:42.016 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:42.016 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:11:42.016 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:11:42.016 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T00:11:42.016 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T00:11:42.016 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T00:11:42.017 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T00:11:42.018 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:42.018 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T00:11:42.018 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:42.018 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T00:11:42.018 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T00:11:42.018 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T00:11:42.021 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T00:11:42.021 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:42.022 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T00:11:30:997230+0000", 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:42.023 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T00:11:30.997976+0000", 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:11:42.024 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (1/15)... 2024-09-11T00:11:44.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:44 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2624820945' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-11T00:11:45.403 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:45.403 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T00:11:45.404 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T00:11:45.405 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:45.406 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T00:11:45.406 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T00:11:45.406 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T00:11:45.406 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T00:11:45.406 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T00:11:45.406 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T00:11:45.406 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T00:11:30:997230+0000", 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T00:11:45.409 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:45.410 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T00:11:30.997976+0000", 2024-09-11T00:11:45.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:11:45.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:45.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T00:11:45.411 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:11:45.411 INFO:teuthology.orchestra.run.smithi103.stdout:mgr not available, waiting (2/15)... 2024-09-11T00:11:46.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: Activating manager daemon x 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: mgrmap e2: x(active, starting, since 0.00500188s) 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: Manager daemon x is now available 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' 2024-09-11T00:11:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:11:46.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' 2024-09-11T00:11:46.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:46 smithi103 ceph-mon[30578]: from='mgr.14100 172.21.15.103:0/2330731199' entity='mgr.x' 2024-09-11T00:11:48.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:48 smithi103 ceph-mon[30578]: mgrmap e3: x(active, since 1.00922s) 2024-09-11T00:11:48.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:48 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1555765545' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T00:11:48.904 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 0 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T00:11:48.905 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T00:11:48.906 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T00:11:30:997230+0000", 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T00:11:48.907 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ], 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T00:11:30.997976+0000", 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout }, 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:11:48.908 INFO:teuthology.orchestra.run.smithi103.stdout:mgr is available 2024-09-11T00:11:49.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:49 smithi103 ceph-mon[30578]: mgrmap e4: x(active, since 2s) 2024-09-11T00:11:50.263 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:50.263 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T00:11:50.263 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout fsid = 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.103:3300,v1:172.21.15.103:6789] 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T00:11:50.264 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T00:11:50.265 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T00:11:50.265 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T00:11:50.265 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T00:11:50.265 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling cephadm module... 2024-09-11T00:11:50.783 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:50 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2946089199' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-11T00:11:51.887 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:51 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/874674538' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-11T00:11:51.887 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: ignoring --setuser ceph since I am not root 2024-09-11T00:11:51.888 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: ignoring --setgroup ceph since I am not root 2024-09-11T00:11:52.848 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:52 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/874674538' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T00:11:52.848 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:52 smithi103 ceph-mon[30578]: mgrmap e5: x(active, since 5s) 2024-09-11T00:11:53.813 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:53 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/331523214' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-11T00:11:54.075 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 5... 2024-09-11T00:11:58.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: Active manager daemon x restarted 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: Activating manager daemon x 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: mgrmap e6: x(active, starting, since 0.00395598s) 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:11:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:57 smithi103 ceph-mon[30578]: Manager daemon x is now available 2024-09-11T00:11:59.284 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:11:59.047+0000 7fb4dec00640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:00:11:59] ENGINE Error in HTTPServer.serve 2024-09-11T00:11:59.284 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: Traceback (most recent call last): 2024-09-11T00:11:59.284 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:11:59.284 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self._connections.run(self.expiration_interval) 2024-09-11T00:11:59.284 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:11:59.284 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self._run(expiration_interval) 2024-09-11T00:11:59.284 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: s = self.context.wrap_socket( 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: return self.sslsocket_class._create( 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self.do_handshake() 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self._sslobj.do_handshake() 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:11:59.285 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: Found migration_current of "None". Setting to last migration. 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T00:11:59.286 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: mgrmap e7: x(active, since 1.00666s) 2024-09-11T00:11:59.287 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:11:59 smithi103 ceph-mon[30578]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T00:11:59.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:11:59.290 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T00:11:59.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T00:11:59.291 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:11:59.291 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 5 is available 2024-09-11T00:11:59.291 INFO:teuthology.orchestra.run.smithi103.stdout:Setting orchestrator backend to cephadm... 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:00 smithi103 ceph-mon[30578]: [11/Sep/2024:00:11:58] ENGINE Bus STARTING 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:00 smithi103 ceph-mon[30578]: [11/Sep/2024:00:11:58] ENGINE Serving on http://172.21.15.103:8765 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:00 smithi103 ceph-mon[30578]: [11/Sep/2024:00:11:59] ENGINE Serving on https://172.21.15.103:7150 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:00 smithi103 ceph-mon[30578]: [11/Sep/2024:00:11:59] ENGINE Bus STARTED 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:00 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:00 smithi103 ceph-mon[30578]: [11/Sep/2024:00:11:59] ENGINE Error in HTTPServer.serve 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-11T00:12:00.096 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:12:00.097 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-11T00:12:00.098 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:12:00.098 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-11T00:12:00.098 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:12:01.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:01 smithi103 ceph-mon[30578]: mgrmap e8: x(active, since 2s) 2024-09-11T00:12:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:01 smithi103 ceph-mon[30578]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:01 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:01 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:02.087 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:02 smithi103 ceph-mon[30578]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:02.092 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T00:12:02.093 INFO:teuthology.orchestra.run.smithi103.stdout:Generating ssh key... 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: Generating public/private rsa key pair. 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: Your identification has been saved in /tmp/tmp8pr7cu_f/key 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: Your public key has been saved in /tmp/tmp8pr7cu_f/key.pub 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: The key fingerprint is: 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: SHA256:9Vm+nG8fujA7BlYVPCLvRQRa3Zkdgn70/1QmjrSGa7U ceph-40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: The key's randomart image is: 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: +---[RSA 3072]----+ 2024-09-11T00:12:03.662 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | o*+o.=| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | .oo B.+.| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | .= = + | 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | . =.*..o| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | S oo=+.oo| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | o..=..oo| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | . .* .+o.| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | ooE ..+| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: | ....o..+| 2024-09-11T00:12:03.663 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: +----[SHA256]-----+ 2024-09-11T00:12:04.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:04 smithi103 ceph-mon[30578]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:04.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:04 smithi103 ceph-mon[30578]: Generating ssh key... 2024-09-11T00:12:04.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:04 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:04.666 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:04 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:05.297 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWUpK9TBtJtj6pMm1es5r0ZiYuFAJzcv+RzhbynU7Z1Cgw2jrqYNvYlRqy2w3BCk+5wTDEGjOLXQYhTjU0F0bs8rQnd38qjcAP3OnbFOxr4VeQu4yRx6GXbx0Qj1iXbtB12tWUCv+LS3utWiktGAoW5WltkHLm22KVwO3P0klFW4/ehzeN4EmsMwWvYLVCAg7XV0LuJsBCZBKjSlWrNSCkj3jSC9YOOeYqVemrGzzSFQEMrTp/D6PNedt8eoaR6FVuOS1jbyQM6ay3nXaCeEg9adUGaTJjtxEBuhL1Emi7e3cf+eB2weaRW6p+YCnv3JCCJsJ545/EKce/J3M0c6diYwcRJNDtRYQjz+itvrfO1pmK0X0O3TBpg//gLxxTEPABDI5pexnmA1VT8HfTiSF6eCnuqFpEwrIdvOjuFfCC/1KmijVknKE1bQ50eSq89J6ACNWX3lxNHt4OHQhyQXHJxZl88B++juIJ8X4hxeugfzIsPzhmqxe8Mgilkrt8abM= ceph-40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:12:05.298 INFO:teuthology.orchestra.run.smithi103.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:12:05.298 INFO:teuthology.orchestra.run.smithi103.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T00:12:05.327 INFO:teuthology.orchestra.run.smithi103.stdout:Adding host smithi103... 2024-09-11T00:12:05.870 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:05 smithi103 ceph-mon[30578]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:06.751 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:06 smithi103 ceph-mon[30578]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "addr": "172.21.15.103", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:07.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:07 smithi103 ceph-mon[30578]: Deploying cephadm binary to smithi103 2024-09-11T00:12:10.774 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Added host 'smithi103' with addr '172.21.15.103' 2024-09-11T00:12:10.775 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mon service... 2024-09-11T00:12:11.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:10 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:10 smithi103 ceph-mon[30578]: Added host smithi103 2024-09-11T00:12:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:10 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:13.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:12 smithi103 ceph-mon[30578]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:13.081 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:12 smithi103 ceph-mon[30578]: Saving service mon spec with placement count:5 2024-09-11T00:12:13.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:12 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:13.085 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T00:12:13.085 INFO:teuthology.orchestra.run.smithi103.stdout:Deploying unmanaged mgr service... 2024-09-11T00:12:14.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:14 smithi103 ceph-mon[30578]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:14.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:14 smithi103 ceph-mon[30578]: Saving service mgr spec with placement count:2 2024-09-11T00:12:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:14 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:14 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:14 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:15.030 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T00:12:17.008 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:16 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1810875126' entity='client.admin' 2024-09-11T00:12:18.571 INFO:teuthology.orchestra.run.smithi103.stdout:Enabling the dashboard module... 2024-09-11T00:12:18.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:18 smithi103 ceph-mon[30578]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:18.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:18 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:18.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:18 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2794270633' entity='client.admin' 2024-09-11T00:12:20.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:20 smithi103 ceph-mon[30578]: from='mgr.14118 172.21.15.103:0/2523224152' entity='mgr.x' 2024-09-11T00:12:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:20 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1873372787' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-11T00:12:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:20 smithi103 ceph-mon[30578]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:20.420 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:20 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: ignoring --setuser ceph since I am not root 2024-09-11T00:12:20.420 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:20 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: ignoring --setgroup ceph since I am not root 2024-09-11T00:12:21.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:21 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1873372787' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T00:12:21.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:21 smithi103 ceph-mon[30578]: mgrmap e9: x(active, since 22s) 2024-09-11T00:12:22.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:22 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/346202703' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-11T00:12:23.033 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:12:23.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T00:12:23.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T00:12:23.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T00:12:23.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T00:12:23.034 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:12:23.034 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for the mgr to restart... 2024-09-11T00:12:23.034 INFO:teuthology.orchestra.run.smithi103.stdout:Waiting for mgr epoch 9... 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: Active manager daemon x restarted 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: Activating manager daemon x 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: mgrmap e10: x(active, starting, since 0.0043921s) 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-11T00:12:26.657 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:26 smithi103 ceph-mon[30578]: Manager daemon x is now available 2024-09-11T00:12:27.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:27.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-11T00:12:27.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-11T00:12:27.574 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-mon[30578]: mgrmap e11: x(active, since 1.00806s) 2024-09-11T00:12:27.893 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:12:27.571+0000 7f7861a00640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:00:12:27] ENGINE Error in HTTPServer.serve 2024-09-11T00:12:27.893 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: Traceback (most recent call last): 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self._connections.run(self.expiration_interval) 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self._run(expiration_interval) 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: s = self.context.wrap_socket( 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:12:27.894 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: return self.sslsocket_class._create( 2024-09-11T00:12:27.895 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:12:27.895 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self.do_handshake() 2024-09-11T00:12:27.895 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:12:27.895 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: self._sslobj.do_handshake() 2024-09-11T00:12:27.895 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:12:27.895 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:12:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:12:28.628 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:28 smithi103 ceph-mon[30578]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T00:12:28.628 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:28 smithi103 ceph-mon[30578]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T00:12:28.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout { 2024-09-11T00:12:28.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T00:12:28.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T00:12:28.633 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout } 2024-09-11T00:12:28.633 INFO:teuthology.orchestra.run.smithi103.stdout:mgr epoch 9 is available 2024-09-11T00:12:28.633 INFO:teuthology.orchestra.run.smithi103.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-11T00:12:29.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:29 smithi103 ceph-mon[30578]: [11/Sep/2024:00:12:27] ENGINE Bus STARTING 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:29 smithi103 ceph-mon[30578]: [11/Sep/2024:00:12:27] ENGINE Serving on http://172.21.15.103:8765 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:29 smithi103 ceph-mon[30578]: [11/Sep/2024:00:12:27] ENGINE Serving on https://172.21.15.103:7150 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:29 smithi103 ceph-mon[30578]: [11/Sep/2024:00:12:27] ENGINE Bus STARTED 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:29 smithi103 ceph-mon[30578]: [11/Sep/2024:00:12:27] ENGINE Error in HTTPServer.serve 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: Traceback (most recent call last): 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._connections.run(self.expiration_interval) 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._run(expiration_interval) 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T00:12:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: s = self.context.wrap_socket( 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: return self.sslsocket_class._create( 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: self.do_handshake() 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: self._sslobj.do_handshake() 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:29 smithi103 ceph-mon[30578]: mgrmap e12: x(active, since 2s) 2024-09-11T00:12:29.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:30.816 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE/DCCAuSgAwIBAgIUE8DoYJxmEg6TRgJ1Rs/G8L7PuZIwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMTAwMTIzMFoXDTM0\nMDkxMjAwMTIzMFowGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjEwMzCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBAJ8SuAiAxy6K36HWwUP/gjthGoRB0U0LjhCA\nO4MLI/nF0EVXvdU5vOX6VELvP+6Y0REgVFuCHj4DwLMhRlvf7yB/fqKFTx0RtZAA\nNYz8Mq+fFlKUSYYLcUYzNriX29bkZyMR1BpNVV8cPMLDT3QXRAYqD+vm4WYgqwiG\niTEgVqbsEhW4FUTz/txzWFMPEIVyhc6Z7PiRb6yjKXtF884eSK8AzwkJ5B4E0/AX\nDrwSdzdMdYCXK7sYGUJo9hfNUx4rWn20qq7oeGgyPguN+EOr+nb1j3RWPLYMkfjt\nHpgYy0Fd3I09IcujAbF768jp0/m+LoMuANBiHcv7nLgbvc60zKFy3DRB8bGbtmft\nazW0h4orrpwnuRnSMOPOLV6ZKo/OhmDGZEweABKXpiuK9zCso7whtx6Y8GtkaUz9\nqZ7lm/FCGkfXC1pNEjO+dRpgXU8vqaS2i86ld0Wui4Eh8Vf6K70IROYg3QGL8LrE\n9RJHIktr+Yz3Op/E+qhSqi83T1XjP63vewadx8VPp9kNSUHebMQZF9CFT4WvYB00\nfepvhJ5B5urFAyAE2lG2cP5ACmQ8cHncaac/siK0SsOHZP1QK9kb6dOlHYX+vSNY\nEk6wU6lfFt2yIpSEQ/1sHCgmBCNCqXX+LPPxvchCj6BLGdEIxWlfrc6HqU4rLsyj\no2SSgqWBAgMBAAGjPzA9MC0GA1UdEQQmMCSCCXNtaXRoaTEwM4IRZGFzaGJvYXJk\nX3NlcnZlcnOHBKwVD2cwDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAgEA\nAONNvsbzRyFG/Pm3K3NZOLPK+36oOi+HTI5kLS8z4Q0IBVnjDU3MpaDym4P5eSl/\nB+gwk99a3CCKVMukf86MU3MCewNZ8FubSoMwqd1Ls5nHWF66wpnA4NbUpJxjr3IL\n21XEQCFrMDYVXf7DBYCZOqTacUTYnioy4imCosag7CGozbNd57bTZcsYFZT07a/2\nMVdN5/n9J4U1SkADzzhdTfyRE6vHG4g+XLulZ6z7DzheRXHD0ZE+Q1CVeCENYnxY\nhwUUqtHQc2/hN0fBojf5aMBXN2DP7yx9nw4nyqQQtGiniUvMskb2N4UIGcSxlMAj\nAQ8X2416WshiX30RFlrcaw91XXles7IkcEyUP0I8k59hbfyMuoOxcpervpY15fIs\nimw2wBQd2JSgjG4PUTB2+whjkT0WLhHFm6JNTGOw1E800bBTYMeH0W4vrIn0KAFB\nGesGQP9uFtgNQKtCxGqkwCgg9jBgM4caXgai0SHCf34nVat8BH0jePUpvKrjqjou\nRRbVKz5WIokTJWvojJVE5aIi/GR1Dr+O5Dwh8FFSbJQE5SitUoeVAJeDdk7z5F66\nOgS80edT8FbIzTjROrXRK/8ppLM1O33WvbT3TFVdoDjDeRDycFJrgOBkdRsLODsL\nX0kQGLcFd0QecXgH3u3ITmuG/DsjfnYAQboUFwbRC88=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKgIBAAKCAgEAnxK4CIDHLorfodbBQ/+CO2EahEHRTQuOEIA7gwsj+cXQRVe9\n1Tm85fpUQu8/7pjRESBUW4IePgPAsyFGW9/vIH9+ooVPHRG1kAA1jPwyr58WUpRJ\nhgtxRjM2uJfb1uRnIxHUGk1VXxw8wsNPdBdEBioP6+bhZiCrCIaJMSBWpuwSFbgV\nRPP+3HNYUw8QhXKFzpns+JFvrKMpe0Xzzh5IrwDPCQnkHgTT8BcOvBJ3N0x1gJcr\nuxgZQmj2F81THitafbSqruh4aDI+C434Q6v6dvWPdFY8tgyR+O0emBjLQV3cjT0h\ny6MBsXvryOnT+b4ugy4A0GIdy/ucuBu9zrTMoXLcNEHxsZu2Z+1rNbSHiiuunCe5\nGdIw484tXpkqj86GYMZkTB4AEpemK4r3MKyjvCG3Hpjwa2RpTP2pnuWb8UIaR9cL\nWk0SM751GmBdTy+ppLaLzqV3Ra6LgSHxV/orvQhE5iDdAYvwusT1EkciS2v5jPc6\nn8T6qFKqLzdPVeM/re97Bp3HxU+n2Q1JQd5sxBkX0IVPha9gHTR96m+EnkHm6sUD\nIATaUbZw/kAKZDxwedxppz+yIrRKw4dk/VAr2Rvp06Udhf69I1gSTrBTqV8W3bIi\nlIRD/WwcKCYEI0Kpdf4s8/G9yEKPoEsZ0QjFaV+tzoepTisuzKOjZJKCpYECAwEA\nAQKCAgAR4y95mU2MkMYz8yu5Yz54QeOS8rm/8agwd9L5WePaPuG8InEwzHTO4Zbf\nDftA78M2q02BGsEe1Royqm+4fhZsL5oEWXw7dxXVEwjyiRuD/x4p/A3Genvq2nBx\nUnZAlF96pJMWwBJ1Bv5YIlFJayJVlR5ENyqqassAwuwjb5D+5TXlZlVR8yP6Eyhf\n8qY3c1jTkiCBFQ2bOp+8AnI3f9VJhIxDZm7V51ITwewAvwzT1Y+y/YzH1Z5lhtsB\ncy/BCTv8PbghwfdkNB4g+35q0ahw/wz6FoZuLDyv4q4ZiVQ1zgGyiijGNtbGZ/ps\nFcsl3rppOt+Rjy7tlKYbV9dAqrkF0xl/NSLncQPEFiI/BQehqXNterQJBmKaWbZ/\ny5QsQ3qHQN5YtRRo3ODNjaPcwf3jxUUo7oPi5GLlQYXQOXdFL6w/+X953lFOkfeL\nDnrhmt4le+66GQkRHJuVYc65gaKc/VblvfAjwkViY8cJgl+q8a993S+Bbw2LtEMS\n8sLheiIcMVm9/dAw84GU9KErXQM1+y0qk1byxI2TZsn8OPQjq+5YfJqw/JKPR2ux\nvu6IrtqgJ309jCsbGhtDlbSoEL+OzKzJFhlmxZtFVo83YkxNX2srvAT6EnDbpjBZ\nS3LQz3ohvNcPSw3g0klwKu+H/tk9+pUTUY8XEm2V4ZsVuisNJwKCAQEA2eeBcV9b\nL6qRYc2O0X0rTrC5jP/VjMKzJ3uc8XIoSZREGhmYoWuTeyU8z3KPLw7dhAzIouIp\nn8lFx6ve4IEnHxy0qZ90efkTQsdP0nKyEk5GExuDG/ui1hD+K8DF8S3sgofdUmDU\nedHxIkxz6xOWmSc6bAezW0yew3T9oFgG3jxUaSSD6uWgyoHaXtZ3zJQKaeykh4xe\nZ/+Pqv/4Y0UoqHF6ScS/GLKVfPazgD1RH419r6wudCY8+0wRF1o5/+mY52suewqa\nHeyrckDK8cop1pHMD8kGwCKTvCZ/RFNGR65JTan+iIFajD8AUKZFRgZBq0iPiLDq\nsRGtov+pMoJCzwKCAQEAuuIsLxJlSuqkglRMOqTXv3G0DR1bC2n2EoIdA0XYPPV9\nqhmP2Sbz9Y6NWN2h7go413iwQYMv/X+LHPFf9dZrnf/LAoEyV3pK1hjr9DQ4WWAD\nJ12JHjT94yLiyiN2ocOW9qzBxgfQp2+DO9Ht4KDEg0HG39ePGumjb+HMYw5vjQ9/\nITfyYPch6G2FcHRaT95ImW9e5jWTb1BMFBUN7VGWkCpxggdZzzeQP0ejdxFdeRHq\nXh+fRTXOydAJnPeLJVYD0p4658/hyV1mwrJW5bb/rFmtFD9kn0JBGDmf7SIU/CEn\nIecV6/LFfSCnVGpO3p2R1awhxt83bcsy3ajp5b/mrwKCAQEAh1f13Hc/6hFlj3vf\n01qNhaja9ysIdw+Pu1xaFNW1FE6VJijz1n888WYlqSwcpzvydsyB6jnTZL3dK7Ij\nEwrlQ7vOsvSpogEE55tohuxakmgvujqCauUXNSME3CQ2TiEaH9FHtJ0UHD48KfOW\npFJcXfAxp1v5GSI0Em8eAXPrgCm0vEHD9ih+gIm7GuDXNChXd4dWgiK7mJjxoWKv\n4ujX4hfpvtwHXc2+cAnDYEm1k4WMpYqqp9YAVo3ep8SQ9dUf9c1q512kcCXKL4Lz\nCs9owS95sexI1xoi7OB0d/nJU0h7m1OVFzBRD38qMO21ZVMJgtvk2yAYBbdXREDL\n1OV3nQKCAQEAkjtB2X9qdSmPmk996nFsRGgjPBPZVuAN5UOckiAH6QMq1nvz/DYn\ntOBbp8qc15d2DNxqBokxdwljAD27GbEzOkmaP5Z6nsqZXlfX4Zku8nhXX2koTtOm\nSiHVFdiUDlXufYyGf39D/JAcPyWwHhGOUyqYBDU2zBc/hN9JP8Ve8iDLGGIfrVIT\nXTy9qlXRCSfk5PKpbjdD61kIfrpO0IED8EDV2qoH00Yl/+39ccDLxF2JZycVSGGn\nMnEv5HwqjAd3LEoVy6f1g8wrCXPk3RDJlMIC1ZIZGSkNqHgGtNXzV6pVZYjdtWec\nsqFnbX1Vw4nGQEzVjTjCtY3HenyesODtDwKCAQEAzl9W483rjMrwNZ9iqQosCiaj\nHfhuJyk4VjuDbyL4dnF6nZBba498CKk701iwT52eX+0BvQ0AO02oIzQTCh5z+021\n81zdJUBkzcH0EZlKVZt3yG+PQdGY7dxnqPYgHQZ1QSM10ALrxJ+4p+fAvJWR3jfp\nVBbSgS6Z6uNiEKO+Mo9eApyT5/pAalqy9UPM6nkB4Nv7FX51M/98EzR7bwE5119j\nTWRUImQYEQSzN/JbHGLgxCtv3v9VrefWA6mUPCLaW0Vc2RD1iaNXw3pTuwNhCudI\n0REzQhHIaAbFJfNsRE5gSPExPlJkL4KWW7hIymEbKWyvsGY2j+o7ulEWPimJow==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-11T00:12:31.093 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:30 smithi103 ceph-mon[30578]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:31.093 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:30 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:32.863 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:32 smithi103 ceph-mon[30578]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:32.863 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:32 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:32.869 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-11T00:12:34.577 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-11T00:12:34.577 INFO:teuthology.orchestra.run.smithi103.stdout:Creating initial admin user... 2024-09-11T00:12:35.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:34 smithi103 ceph-mon[30578]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:35.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:36.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:35 smithi103 ceph-mon[30578]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:36.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:36.225 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$z2wQ//uW3aDIa/mMaVQnyeHxor8s9IGLZk4mjqXnS8KuntfFlOSWG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726013555, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T00:12:36.225 INFO:teuthology.orchestra.run.smithi103.stdout:Fetching dashboard port number... 2024-09-11T00:12:37.316 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:37 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/639311391' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-11T00:12:37.625 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T00:12:37.652 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T00:12:37.652 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout disabled 2024-09-11T00:12:37.666 INFO:teuthology.orchestra.run.smithi103.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T00:12:37.666 INFO:teuthology.orchestra.run.smithi103.stdout:systemctl: stdout inactive 2024-09-11T00:12:37.666 INFO:teuthology.orchestra.run.smithi103.stdout:firewalld.service is not enabled 2024-09-11T00:12:37.666 INFO:teuthology.orchestra.run.smithi103.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T00:12:37.670 INFO:teuthology.orchestra.run.smithi103.stdout:Ceph Dashboard is now available at: 2024-09-11T00:12:37.671 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:37.671 INFO:teuthology.orchestra.run.smithi103.stdout: URL: https://smithi103.front.sepia.ceph.com:8443/ 2024-09-11T00:12:37.671 INFO:teuthology.orchestra.run.smithi103.stdout: User: admin 2024-09-11T00:12:37.671 INFO:teuthology.orchestra.run.smithi103.stdout: Password: vdl77nqr0m 2024-09-11T00:12:37.671 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:37.671 INFO:teuthology.orchestra.run.smithi103.stdout:Saving cluster configuration to /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config directory 2024-09-11T00:12:39.172 INFO:teuthology.orchestra.run.smithi103.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T00:12:39.172 INFO:teuthology.orchestra.run.smithi103.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T00:12:39.172 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.172 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:12:39.172 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.172 INFO:teuthology.orchestra.run.smithi103.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T00:12:39.172 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: ceph telemetry on 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout:For more information see: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:39.173 INFO:teuthology.orchestra.run.smithi103.stdout:Bootstrap complete. 2024-09-11T00:12:39.230 INFO:tasks.cephadm:Fetching config... 2024-09-11T00:12:39.231 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:12:39.231 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T00:12:39.271 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T00:12:39.271 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:12:39.271 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T00:12:39.348 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T00:12:39.348 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:12:39.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T00:12:39.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:39 smithi103 ceph-mon[30578]: mgrmap e13: x(active, since 12s) 2024-09-11T00:12:39.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:39 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4250908019' entity='client.admin' 2024-09-11T00:12:39.444 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T00:12:39.444 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:12:39.444 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T00:12:39.520 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T00:12:39.520 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWUpK9TBtJtj6pMm1es5r0ZiYuFAJzcv+RzhbynU7Z1Cgw2jrqYNvYlRqy2w3BCk+5wTDEGjOLXQYhTjU0F0bs8rQnd38qjcAP3OnbFOxr4VeQu4yRx6GXbx0Qj1iXbtB12tWUCv+LS3utWiktGAoW5WltkHLm22KVwO3P0klFW4/ehzeN4EmsMwWvYLVCAg7XV0LuJsBCZBKjSlWrNSCkj3jSC9YOOeYqVemrGzzSFQEMrTp/D6PNedt8eoaR6FVuOS1jbyQM6ay3nXaCeEg9adUGaTJjtxEBuhL1Emi7e3cf+eB2weaRW6p+YCnv3JCCJsJ545/EKce/J3M0c6diYwcRJNDtRYQjz+itvrfO1pmK0X0O3TBpg//gLxxTEPABDI5pexnmA1VT8HfTiSF6eCnuqFpEwrIdvOjuFfCC/1KmijVknKE1bQ50eSq89J6ACNWX3lxNHt4OHQhyQXHJxZl88B++juIJ8X4hxeugfzIsPzhmqxe8Mgilkrt8abM= ceph-40f10072-6fd2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T00:12:39.641 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWUpK9TBtJtj6pMm1es5r0ZiYuFAJzcv+RzhbynU7Z1Cgw2jrqYNvYlRqy2w3BCk+5wTDEGjOLXQYhTjU0F0bs8rQnd38qjcAP3OnbFOxr4VeQu4yRx6GXbx0Qj1iXbtB12tWUCv+LS3utWiktGAoW5WltkHLm22KVwO3P0klFW4/ehzeN4EmsMwWvYLVCAg7XV0LuJsBCZBKjSlWrNSCkj3jSC9YOOeYqVemrGzzSFQEMrTp/D6PNedt8eoaR6FVuOS1jbyQM6ay3nXaCeEg9adUGaTJjtxEBuhL1Emi7e3cf+eB2weaRW6p+YCnv3JCCJsJ545/EKce/J3M0c6diYwcRJNDtRYQjz+itvrfO1pmK0X0O3TBpg//gLxxTEPABDI5pexnmA1VT8HfTiSF6eCnuqFpEwrIdvOjuFfCC/1KmijVknKE1bQ50eSq89J6ACNWX3lxNHt4OHQhyQXHJxZl88B++juIJ8X4hxeugfzIsPzhmqxe8Mgilkrt8abM= ceph-40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:12:39.666 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWUpK9TBtJtj6pMm1es5r0ZiYuFAJzcv+RzhbynU7Z1Cgw2jrqYNvYlRqy2w3BCk+5wTDEGjOLXQYhTjU0F0bs8rQnd38qjcAP3OnbFOxr4VeQu4yRx6GXbx0Qj1iXbtB12tWUCv+LS3utWiktGAoW5WltkHLm22KVwO3P0klFW4/ehzeN4EmsMwWvYLVCAg7XV0LuJsBCZBKjSlWrNSCkj3jSC9YOOeYqVemrGzzSFQEMrTp/D6PNedt8eoaR6FVuOS1jbyQM6ay3nXaCeEg9adUGaTJjtxEBuhL1Emi7e3cf+eB2weaRW6p+YCnv3JCCJsJ545/EKce/J3M0c6diYwcRJNDtRYQjz+itvrfO1pmK0X0O3TBpg//gLxxTEPABDI5pexnmA1VT8HfTiSF6eCnuqFpEwrIdvOjuFfCC/1KmijVknKE1bQ50eSq89J6ACNWX3lxNHt4OHQhyQXHJxZl88B++juIJ8X4hxeugfzIsPzhmqxe8Mgilkrt8abM= ceph-40f10072-6fd2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T00:12:39.706 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWUpK9TBtJtj6pMm1es5r0ZiYuFAJzcv+RzhbynU7Z1Cgw2jrqYNvYlRqy2w3BCk+5wTDEGjOLXQYhTjU0F0bs8rQnd38qjcAP3OnbFOxr4VeQu4yRx6GXbx0Qj1iXbtB12tWUCv+LS3utWiktGAoW5WltkHLm22KVwO3P0klFW4/ehzeN4EmsMwWvYLVCAg7XV0LuJsBCZBKjSlWrNSCkj3jSC9YOOeYqVemrGzzSFQEMrTp/D6PNedt8eoaR6FVuOS1jbyQM6ay3nXaCeEg9adUGaTJjtxEBuhL1Emi7e3cf+eB2weaRW6p+YCnv3JCCJsJ545/EKce/J3M0c6diYwcRJNDtRYQjz+itvrfO1pmK0X0O3TBpg//gLxxTEPABDI5pexnmA1VT8HfTiSF6eCnuqFpEwrIdvOjuFfCC/1KmijVknKE1bQ50eSq89J6ACNWX3lxNHt4OHQhyQXHJxZl88B++juIJ8X4hxeugfzIsPzhmqxe8Mgilkrt8abM= ceph-40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:12:39.720 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWUpK9TBtJtj6pMm1es5r0ZiYuFAJzcv+RzhbynU7Z1Cgw2jrqYNvYlRqy2w3BCk+5wTDEGjOLXQYhTjU0F0bs8rQnd38qjcAP3OnbFOxr4VeQu4yRx6GXbx0Qj1iXbtB12tWUCv+LS3utWiktGAoW5WltkHLm22KVwO3P0klFW4/ehzeN4EmsMwWvYLVCAg7XV0LuJsBCZBKjSlWrNSCkj3jSC9YOOeYqVemrGzzSFQEMrTp/D6PNedt8eoaR6FVuOS1jbyQM6ay3nXaCeEg9adUGaTJjtxEBuhL1Emi7e3cf+eB2weaRW6p+YCnv3JCCJsJ545/EKce/J3M0c6diYwcRJNDtRYQjz+itvrfO1pmK0X0O3TBpg//gLxxTEPABDI5pexnmA1VT8HfTiSF6eCnuqFpEwrIdvOjuFfCC/1KmijVknKE1bQ50eSq89J6ACNWX3lxNHt4OHQhyQXHJxZl88B++juIJ8X4hxeugfzIsPzhmqxe8Mgilkrt8abM= ceph-40f10072-6fd2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T00:12:39.761 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDWUpK9TBtJtj6pMm1es5r0ZiYuFAJzcv+RzhbynU7Z1Cgw2jrqYNvYlRqy2w3BCk+5wTDEGjOLXQYhTjU0F0bs8rQnd38qjcAP3OnbFOxr4VeQu4yRx6GXbx0Qj1iXbtB12tWUCv+LS3utWiktGAoW5WltkHLm22KVwO3P0klFW4/ehzeN4EmsMwWvYLVCAg7XV0LuJsBCZBKjSlWrNSCkj3jSC9YOOeYqVemrGzzSFQEMrTp/D6PNedt8eoaR6FVuOS1jbyQM6ay3nXaCeEg9adUGaTJjtxEBuhL1Emi7e3cf+eB2weaRW6p+YCnv3JCCJsJ545/EKce/J3M0c6diYwcRJNDtRYQjz+itvrfO1pmK0X0O3TBpg//gLxxTEPABDI5pexnmA1VT8HfTiSF6eCnuqFpEwrIdvOjuFfCC/1KmijVknKE1bQ50eSq89J6ACNWX3lxNHt4OHQhyQXHJxZl88B++juIJ8X4hxeugfzIsPzhmqxe8Mgilkrt8abM= ceph-40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:12:39.775 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T00:12:40.137 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:12:41.584 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T00:12:41.584 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T00:12:42.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:41 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/842913134' entity='client.admin' 2024-09-11T00:12:42.191 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:12:43.194 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:43.194 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:43.194 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"} : dispatch 2024-09-11T00:12:43.194 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:43.194 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:43.194 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:44.257 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-09-11T00:12:44.257 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:12:44.257 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-09-11T00:12:44.278 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:12:44.278 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:12:44.336 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-09-11T00:12:44.336 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch host add smithi117 2024-09-11T00:12:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:44 smithi103 ceph-mon[30578]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:12:44.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:12:44.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:44 smithi103 ceph-mon[30578]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-11T00:12:45.234 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:12:45.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:45 smithi103 ceph-mon[30578]: Updating smithi103:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:12:45.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:45 smithi103 ceph-mon[30578]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:12:45.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:45.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:45.496 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:46.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:46 smithi103 ceph-mon[30578]: Updating smithi103:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:12:47.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:47 smithi103 ceph-mon[30578]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:47.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:47 smithi103 ceph-mon[30578]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:48.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:48 smithi103 ceph-mon[30578]: Deploying cephadm binary to smithi117 2024-09-11T00:12:49.086 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-09-11T00:12:49.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:49 smithi103 ceph-mon[30578]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:49.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:49.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:49.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T00:12:50.184 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:12:50.625 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:50 smithi103 ceph-mon[30578]: Added host smithi117 2024-09-11T00:12:50.625 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:50.625 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:50.934 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:50.934 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-09-11T00:12:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:51 smithi103 ceph-mon[30578]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:51.661 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:51 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:51.698 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-09-11T00:12:51.698 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:12:51.698 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-09-11T00:12:51.717 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:12:51.717 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:12:51.775 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-09-11T00:12:51.775 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch host add smithi183 2024-09-11T00:12:52.132 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:12:52.434 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:52 smithi103 ceph-mon[30578]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:12:53.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"} : dispatch 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:12:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:53 smithi103 ceph-mon[30578]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-11T00:12:55.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: Updating smithi117:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:12:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:12:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: Updating smithi117:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:12:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: Deploying cephadm binary to smithi183 2024-09-11T00:12:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:55 smithi103 ceph-mon[30578]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:56.263 INFO:teuthology.orchestra.run.smithi103.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-09-11T00:12:57.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T00:12:57.426 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:12:57.565 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:57 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:57.565 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:57 smithi103 ceph-mon[30578]: Added host smithi183 2024-09-11T00:12:57.565 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:57 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:12:57.565 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:57 smithi103 ceph-mon[30578]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:57.566 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:57 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:57.566 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:57 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:58.256 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:12:58.256 INFO:teuthology.orchestra.run.smithi103.stdout:[{"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-09-11T00:12:58.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:12:58.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:58 smithi103 ceph-mon[30578]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:12:58.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:12:58 smithi103 ceph-mon[30578]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:12:58.996 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T00:12:58.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T00:12:59.352 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:00.147 INFO:teuthology.orchestra.run.smithi103.stderr:adjusted tunables profile to default 2024-09-11T00:13:00.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:00 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2584663541' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-11T00:13:00.940 INFO:tasks.cephadm:Adding mon.a on smithi103 2024-09-11T00:13:00.940 INFO:tasks.cephadm:Adding mon.b on smithi117 2024-09-11T00:13:00.940 INFO:tasks.cephadm:Adding mon.c on smithi183 2024-09-11T00:13:00.940 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi103:172.21.15.103=a;smithi117:172.21.15.117=b;smithi183:172.21.15.183=c' 2024-09-11T00:13:01.183 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:01.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2584663541' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-11T00:13:01.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:01.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:01.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:01.787 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mon update... 2024-09-11T00:13:02.388 DEBUG:teuthology.orchestra.run.smithi117:mon.b> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T00:13:02.391 DEBUG:teuthology.orchestra.run.smithi183:mon.c> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T00:13:02.393 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T00:13:02.394 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T00:13:02.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: Updating smithi183:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:02.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: Updating smithi183:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:02.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:03.399 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:03.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:03 smithi103 ceph-mon[30578]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi103:172.21.15.103=a;smithi117:172.21.15.117=b;smithi183:172.21.15.183=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:13:03.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:03 smithi103 ceph-mon[30578]: Saving service mon spec with placement smithi103:172.21.15.103=a;smithi117:172.21.15.117=b;smithi183:172.21.15.183=c;count:3 2024-09-11T00:13:03.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:03 smithi103 ceph-mon[30578]: Deploying daemon mon.c on smithi183 2024-09-11T00:13:03.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:03 smithi103 ceph-mon[30578]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:04.942 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:13:04.942 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":1,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","modified":"2024-09-11T00:11:26.844308Z","created":"2024-09-11T00:11:26.844308Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T00:13:04.942 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 1 2024-09-11T00:13:05.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:05 smithi103 ceph-mon[30578]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:05.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:05 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/1622837265' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:13:06.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:06 smithi183 ceph-mon[35934]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-11T00:13:06.722 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T00:13:06.722 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T00:13:06.975 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: mon.a calling monitor election 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: mon.c calling monitor election 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: monmap epoch 2 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: last_changed 2024-09-11T00:13:06.122334+0000 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: created 2024-09-11T00:11:26.844308+0000 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: min_mon_release 19 (squid) 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: election_strategy: 1 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: fsmap 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: mgrmap e13: x(active, since 44s) 2024-09-11T00:13:11.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: overall HEALTH_OK 2024-09-11T00:13:11.533 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:11.533 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:11.533 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:11.533 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:11.533 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:11.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:13:11.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: mon.a calling monitor election 2024-09-11T00:13:11.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:11.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: mon.c calling monitor election 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: monmap epoch 2 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: last_changed 2024-09-11T00:13:06.122334+0000 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: created 2024-09-11T00:11:26.844308+0000 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: min_mon_release 19 (squid) 2024-09-11T00:13:11.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: election_strategy: 1 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: fsmap 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: mgrmap e13: x(active, since 44s) 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: overall HEALTH_OK 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:11.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:12.066 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:13:12.067 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","modified":"2024-09-11T00:13:06.122334Z","created":"2024-09-11T00:11:26.844308Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T00:13:12.067 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-09-11T00:13:12.424 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:12 smithi183 ceph-mon[35934]: Deploying daemon mon.b on smithi117 2024-09-11T00:13:12.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:12 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/3010835621' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:13:12.425 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:12 smithi103 ceph-mon[30578]: Deploying daemon mon.b on smithi117 2024-09-11T00:13:12.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:12 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/3010835621' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:13:12.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:13.327 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:13 smithi103 ceph-mon[30578]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:13.327 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:13:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:13:13.122+0000 7f78a8e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T00:13:13.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:13 smithi183 ceph-mon[35934]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:13.729 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T00:13:13.729 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T00:13:13.981 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:13:14.644 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:13:14.644 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":2,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","modified":"2024-09-11T00:13:06.122334Z","created":"2024-09-11T00:11:26.844308Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T00:13:14.644 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 2 2024-09-11T00:13:15.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:14 smithi117 systemd[1]: Started Ceph mon.b for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:13:16.252 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T00:13:16.254 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T00:13:16.505 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: mon.a calling monitor election 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: mon.c calling monitor election 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: mon.b calling monitor election 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: monmap epoch 3 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: last_changed 2024-09-11T00:13:14.906910+0000 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: created 2024-09-11T00:11:26.844308+0000 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: min_mon_release 19 (squid) 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: election_strategy: 1 2024-09-11T00:13:20.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: fsmap 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: mgrmap e13: x(active, since 53s) 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: overall HEALTH_OK 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.283 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:13:20.390 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":3,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","modified":"2024-09-11T00:13:14.906910Z","created":"2024-09-11T00:11:26.844308Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T00:13:20.390 INFO:teuthology.orchestra.run.smithi183.stderr:dumped monmap epoch 3 2024-09-11T00:13:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:13:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: mon.a calling monitor election 2024-09-11T00:13:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: mon.c calling monitor election 2024-09-11T00:13:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: mon.b calling monitor election 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: monmap epoch 3 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: last_changed 2024-09-11T00:13:14.906910+0000 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: created 2024-09-11T00:11:26.844308+0000 2024-09-11T00:13:20.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: min_mon_release 19 (squid) 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: election_strategy: 1 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: fsmap 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: mgrmap e13: x(active, since 53s) 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: overall HEALTH_OK 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:13:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: mon.a calling monitor election 2024-09-11T00:13:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:13:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: mon.c calling monitor election 2024-09-11T00:13:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: mon.b calling monitor election 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: monmap epoch 3 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: last_changed 2024-09-11T00:13:14.906910+0000 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: created 2024-09-11T00:11:26.844308+0000 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: min_mon_release 19 (squid) 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: election_strategy: 1 2024-09-11T00:13:20.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: 0: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.a 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: 1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.c 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: 2: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: fsmap 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: mgrmap e13: x(active, since 53s) 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: overall HEALTH_OK 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:20.439 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:21.015 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T00:13:21.015 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T00:13:21.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:21.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:20 smithi183 ceph-mon[35934]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:21.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:20 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/2655684623' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:13:21.265 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:21.388 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:21.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:21.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:20 smithi103 ceph-mon[30578]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:21.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:20 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/2655684623' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:13:21.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:20 smithi117 ceph-mon[35525]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:20 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/2655684623' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-11T00:13:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:13:22.169 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:13:21 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:13:21.906+0000 7f78a8e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T00:13:22.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:22.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:22.288 INFO:teuthology.orchestra.run.smithi103.stdout:# minimal ceph.conf for 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:13:22.288 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-09-11T00:13:22.288 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:13:22.288 INFO:teuthology.orchestra.run.smithi103.stdout: mon_host = [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-09-11T00:13:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:22.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:22.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:22.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:22.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:23.084 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T00:13:23.084 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:13:23.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T00:13:23.141 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:13:23.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:13:23.242 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:13:23.242 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T00:13:23.273 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:13:23.273 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:13:23.343 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:13:23.343 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T00:13:23.375 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:13:23.375 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: Updating smithi117:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: Updating smithi183:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: Updating smithi103:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4061917913' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:23.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:23.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:23.444 INFO:tasks.cephadm:Adding mgr.x on smithi103 2024-09-11T00:13:23.444 INFO:tasks.cephadm:Adding mgr.y on smithi117 2024-09-11T00:13:23.444 INFO:tasks.cephadm:Adding mgr.z on smithi183 2024-09-11T00:13:23.445 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi103=x;smithi117=y;smithi183=z' 2024-09-11T00:13:23.508 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: Updating smithi117:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: Updating smithi183:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: Updating smithi103:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4061917913' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:23.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:23.510 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:23.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: Updating smithi103:/etc/ceph/ceph.conf 2024-09-11T00:13:23.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-11T00:13:23.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: Updating smithi117:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: Updating smithi183:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: Updating smithi103:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4061917913' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:23.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:23.726 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:13:24.372 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mgr update... 2024-09-11T00:13:24.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:24 smithi183 ceph-mon[35934]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T00:13:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:24 smithi183 ceph-mon[35934]: Reconfiguring daemon mon.a on smithi103 2024-09-11T00:13:24.542 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:24 smithi103 ceph-mon[30578]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T00:13:24.542 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:24 smithi103 ceph-mon[30578]: Reconfiguring daemon mon.a on smithi103 2024-09-11T00:13:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:24 smithi117 ceph-mon[35525]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T00:13:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:24 smithi117 ceph-mon[35525]: Reconfiguring daemon mon.a on smithi103 2024-09-11T00:13:24.876 DEBUG:teuthology.orchestra.run.smithi117:mgr.y> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T00:13:24.879 DEBUG:teuthology.orchestra.run.smithi183:mgr.z> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T00:13:24.919 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-11T00:13:24.919 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-11T00:13:24.923 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-11T00:13:24.923 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-11T00:13:24.969 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-11T00:13:24.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-11T00:13:25.017 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-11T00:13:25.017 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-11T00:13:25.063 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-11T00:13:25.064 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-11T00:13:25.110 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-11T00:13:25.111 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-11T00:13:25.157 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-11T00:13:25.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-11T00:13:25.208 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-11T00:13:25.209 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-11T00:13:25.265 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-11T00:13:25.265 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-11T00:13:25.317 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-11T00:13:25.317 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-11T00:13:25.369 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-11T00:13:25.369 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-11T00:13:25.427 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-11T00:13:25.428 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-11T00:13:25.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:25 smithi103 ceph-mon[30578]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:25.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:25.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:25.468 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:25.481 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-11T00:13:25.481 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-11T00:13:25.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:25 smithi183 ceph-mon[35934]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:25.535 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-11T00:13:25.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-11T00:13:25.589 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-11T00:13:25.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-11T00:13:25.654 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-11T00:13:25.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-11T00:13:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:25 smithi117 ceph-mon[35525]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:25.731 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-11T00:13:25.731 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-11T00:13:25.787 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-11T00:13:25.787 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-11T00:13:25.848 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-11T00:13:25.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-11T00:13:25.911 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-11T00:13:25.911 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-11T00:13:25.968 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-11T00:13:25.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-11T00:13:26.032 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-11T00:13:26.033 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-11T00:13:26.087 DEBUG:tasks.cephadm:set 22 configs 2024-09-11T00:13:26.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi103') 2024-09-11T00:13:26.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:26 smithi183 ceph-mon[35934]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi117=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:13:26.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:26 smithi183 ceph-mon[35934]: Saving service mgr spec with placement smithi103=x;smithi117=y;smithi183=z;count:3 2024-09-11T00:13:26.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:26 smithi183 ceph-mon[35934]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T00:13:26.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:26 smithi183 ceph-mon[35934]: Reconfiguring daemon mgr.x on smithi103 2024-09-11T00:13:26.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:26 smithi103 ceph-mon[30578]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi117=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:13:26.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:26 smithi103 ceph-mon[30578]: Saving service mgr spec with placement smithi103=x;smithi117=y;smithi183=z;count:3 2024-09-11T00:13:26.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:26 smithi103 ceph-mon[30578]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T00:13:26.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:26 smithi103 ceph-mon[30578]: Reconfiguring daemon mgr.x on smithi103 2024-09-11T00:13:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:26 smithi117 ceph-mon[35525]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi103=x;smithi117=y;smithi183=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:13:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:26 smithi117 ceph-mon[35525]: Saving service mgr spec with placement smithi103=x;smithi117=y;smithi183=z;count:3 2024-09-11T00:13:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:26 smithi117 ceph-mon[35525]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T00:13:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:26 smithi117 ceph-mon[35525]: Reconfiguring daemon mgr.x on smithi103 2024-09-11T00:13:27.058 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.065 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.074 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.075 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.116 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.129 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.138 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.143 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.273 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.293 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.300 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.311 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.332 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:27.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:27 smithi103 ceph-mon[30578]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:27.528 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:27.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:27.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:27.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:27.529 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:27.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:27 smithi183 ceph-mon[35934]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:27.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:27.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:27.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:27.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:27 smithi117 ceph-mon[35525]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:27.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:27.604 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:27.605 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:27.605 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:27.605 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:28.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:28 smithi103 ceph-mon[30578]: Reconfiguring mon.b (monmap changed)... 2024-09-11T00:13:28.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:28 smithi103 ceph-mon[30578]: Reconfiguring daemon mon.b on smithi117 2024-09-11T00:13:28.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:28.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:28.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:28.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:28.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:28.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 -- 172.21.15.103:0/142063345 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bb803f5c0 msgr2=0x7f0bb8041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 --2- 172.21.15.103:0/142063345 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bb803f5c0 0x7f0bb8041a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0095f0 tx=0x7f0bbc005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 -- 172.21.15.103:0/142063345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bd4104aa0 msgr2=0x7f0bd419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 --2- 172.21.15.103:0/142063345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bd4104aa0 0x7f0bd419a0f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400e9f0 tx=0x7f0bc400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 -- 172.21.15.103:0/142063345 shutdown_connections 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 --2- 172.21.15.103:0/142063345 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bb803f5c0 0x7f0bb8041a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 --2- 172.21.15.103:0/142063345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd41065c0 0x7f0bd419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 --2- 172.21.15.103:0/142063345 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0bd4105c00 0x7f0bd419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 --2- 172.21.15.103:0/142063345 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bd4104aa0 0x7f0bd419a0f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 -- 172.21.15.103:0/142063345 >> 172.21.15.103:0/142063345 conn(0x7f0bd4100370 msgr2=0x7f0bd4101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:28.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 -- 172.21.15.103:0/142063345 shutdown_connections 2024-09-11T00:13:28.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.516+0000 7f0bd3200640 1 -- 172.21.15.103:0/142063345 wait complete. 2024-09-11T00:13:28.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:28 smithi183 ceph-mon[35934]: Reconfiguring mon.b (monmap changed)... 2024-09-11T00:13:28.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:28 smithi183 ceph-mon[35934]: Reconfiguring daemon mon.b on smithi117 2024-09-11T00:13:28.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:28.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:28.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:28 smithi117 ceph-mon[35525]: Reconfiguring mon.b (monmap changed)... 2024-09-11T00:13:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:28 smithi117 ceph-mon[35525]: Reconfiguring daemon mon.b on smithi117 2024-09-11T00:13:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-11T00:13:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-11T00:13:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:28.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 -- 172.21.15.103:0/1690465973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634106780 msgr2=0x7f563410d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 --2- 172.21.15.103:0/1690465973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634106780 0x7f563410d100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f562c009930 tx=0x7f562c02a420 comp rx=0 tx=0).stop 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 -- 172.21.15.103:0/1690465973 shutdown_connections 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 --2- 172.21.15.103:0/1690465973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634106780 0x7f563410d100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 --2- 172.21.15.103:0/1690465973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5634105dc0 0x7f5634106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 --2- 172.21.15.103:0/1690465973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 0x7f5634105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 -- 172.21.15.103:0/1690465973 >> 172.21.15.103:0/1690465973 conn(0x7f5634100510 msgr2=0x7f5634102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.775+0000 7f5638c00640 1 -- 172.21.15.103:0/1690465973 shutdown_connections 2024-09-11T00:13:28.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.776+0000 7f5638c00640 1 -- 172.21.15.103:0/1690465973 wait complete. 2024-09-11T00:13:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.776+0000 7f5638c00640 1 Processor -- start 2024-09-11T00:13:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5638c00640 1 -- start start 2024-09-11T00:13:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5638c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 0x7f563419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:28.780 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5638c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634105dc0 0x7f563419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5638c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5634106780 0x7f563419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5638c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f563410fc70 con 0x7f5634105dc0 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5638c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f563410faf0 con 0x7f5634106780 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5638c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f563410fdf0 con 0x7f5634104c60 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5631e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 0x7f563419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5629400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634105dc0 0x7f563419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5631e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 0x7f563419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:38144/0 (socket says 172.21.15.103:38144) 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5631e00640 1 -- 172.21.15.103:0/1834650063 learned_addr learned my addr 172.21.15.103:0/1834650063 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.777+0000 7f5632800640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5634106780 0x7f563419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5631e00640 1 -- 172.21.15.103:0/1834650063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5634106780 msgr2=0x7f563419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:28.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5631e00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5634106780 0x7f563419b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5631e00640 1 -- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634105dc0 msgr2=0x7f563419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5631e00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634105dc0 0x7f563419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5631e00640 1 -- 172.21.15.103:0/1834650063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563419bf70 con 0x7f5634104c60 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5629400640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634105dc0 0x7f563419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5631e00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 0x7f563419a440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f562400db80 tx=0x7f562400b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f562b400640 1 -- 172.21.15.103:0/1834650063 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624017070 con 0x7f5634104c60 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f562b400640 1 -- 172.21.15.103:0/1834650063 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(22 keys) ==== 983+0+0 (secure 0 0 0) 0x7f5624004020 con 0x7f5634104c60 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f562b400640 1 -- 172.21.15.103:0/1834650063 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562400be60 con 0x7f5634104c60 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563419c260 con 0x7f5634104c60 2024-09-11T00:13:28.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.778+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341a2940 con 0x7f5634104c60 2024-09-11T00:13:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.779+0000 7f562b400640 1 -- 172.21.15.103:0/1834650063 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f5624023060 con 0x7f5634104c60 2024-09-11T00:13:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.779+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634105060 con 0x7f5634104c60 2024-09-11T00:13:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.780+0000 7f562b400640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f561803f510 0x7f56180419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.780+0000 7f562b400640 1 -- 172.21.15.103:0/1834650063 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5624053ad0 con 0x7f5634104c60 2024-09-11T00:13:28.783 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.780+0000 7f5629400640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f561803f510 0x7f56180419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:28.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.781+0000 7f5629400640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f561803f510 0x7f56180419d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f561c005d00 tx=0x7f561c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:28.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.785+0000 7f562b400640 1 -- 172.21.15.103:0/1834650063 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5624055e20 con 0x7f5634104c60 2024-09-11T00:13:28.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.974+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f563410b800 con 0x7f5634104c60 2024-09-11T00:13:28.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.981+0000 7f562b400640 1 -- 172.21.15.103:0/1834650063 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v11)=0 v11) ==== 106+0+0 (secure 0 0 0) 0x7f5624010ca0 con 0x7f5634104c60 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f561803f510 msgr2=0x7f56180419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f561803f510 0x7f56180419d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f561c005d00 tx=0x7f561c00b410 comp rx=0 tx=0).stop 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 msgr2=0x7f563419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 0x7f563419a440 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f562400db80 tx=0x7f562400b630 comp rx=0 tx=0).stop 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 shutdown_connections 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f561803f510 0x7f56180419d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5634106780 0x7f563419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5634105dc0 0x7f563419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 --2- 172.21.15.103:0/1834650063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5634104c60 0x7f563419a440 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 >> 172.21.15.103:0/1834650063 conn(0x7f5634100510 msgr2=0x7f56341021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:28.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 shutdown_connections 2024-09-11T00:13:28.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:28.984+0000 7f5638c00640 1 -- 172.21.15.103:0/1834650063 wait complete. 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 -- 172.21.15.103:0/3775985868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e41063b0 msgr2=0x7f01e4111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 --2- 172.21.15.103:0/3775985868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e41063b0 0x7f01e4111070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009ca0 tx=0x7f01dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 -- 172.21.15.103:0/3775985868 shutdown_connections 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 --2- 172.21.15.103:0/3775985868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e41063b0 0x7f01e4111070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 --2- 172.21.15.103:0/3775985868 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01e4075b20 0x7f01e4075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 --2- 172.21.15.103:0/3775985868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01e4076f80 0x7f01e40755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 -- 172.21.15.103:0/3775985868 >> 172.21.15.103:0/3775985868 conn(0x7f01e40fe180 msgr2=0x7f01e41005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:29.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 -- 172.21.15.103:0/3775985868 shutdown_connections 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.043+0000 7f01e3200640 1 -- 172.21.15.103:0/3775985868 wait complete. 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 Processor -- start 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 -- start start 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e4075b20 0x7f01e419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01e4076f80 0x7f01e419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01e419f7e0 0x7f01e419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4113ee0 con 0x7f01e4075b20 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4113d60 con 0x7f01e419f7e0 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4114060 con 0x7f01e4076f80 2024-09-11T00:13:29.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.044+0000 7f01e1e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e4075b20 0x7f01e419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e4075b20 0x7f01e419e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36526/0 (socket says 172.21.15.103:36526) 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 -- 172.21.15.103:0/653580328 learned_addr learned my addr 172.21.15.103:0/653580328 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1400640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01e4076f80 0x7f01e419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e2800640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01e419f7e0 0x7f01e419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 -- 172.21.15.103:0/653580328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01e4076f80 msgr2=0x7f01e419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01e4076f80 0x7f01e419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 -- 172.21.15.103:0/653580328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01e419f7e0 msgr2=0x7f01e419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01e419f7e0 0x7f01e419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 -- 172.21.15.103:0/653580328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e41a0460 con 0x7f01e4075b20 2024-09-11T00:13:29.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1400640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01e4076f80 0x7f01e419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.045+0000 7f01e1e00640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e4075b20 0x7f01e419e550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f01d400d950 tx=0x7f01d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.046+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4017030 con 0x7f01e4075b20 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.046+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(22 keys) ==== 983+0+0 (secure 0 0 0) 0x7f01d4004020 con 0x7f01e4075b20 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.046+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41a6880 con 0x7f01e4075b20 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.046+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d40053b0 con 0x7f01e4075b20 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.046+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41a6d80 con 0x7f01e4075b20 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.047+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e4075fa0 con 0x7f01e4075b20 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.047+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f01d4009a30 con 0x7f01e4075b20 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.047+0000 7f01daa00640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f01c803f5c0 0x7f01c8041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:29.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.047+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f01d4053760 con 0x7f01e4075b20 2024-09-11T00:13:29.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.048+0000 7f01e1400640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f01c803f5c0 0x7f01c8041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:29.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.048+0000 7f01e1400640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f01c803f5c0 0x7f01c8041a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009fc0 tx=0x7f01d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:29.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.054+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f01d40043d0 con 0x7f01e4075b20 2024-09-11T00:13:29.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.244+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f01e4104930 con 0x7f01e4075b20 2024-09-11T00:13:29.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.251+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f01d401ad70 con 0x7f01e4075b20 2024-09-11T00:13:29.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.252+0000 7f01daa00640 1 -- 172.21.15.103:0/653580328 <== mon.0 v2:172.21.15.103:3300/0 8 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f01d401a8e0 con 0x7f01e4075b20 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f01c803f5c0 msgr2=0x7f01c8041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f01c803f5c0 0x7f01c8041a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009fc0 tx=0x7f01d0005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e4075b20 msgr2=0x7f01e419e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e4075b20 0x7f01e419e550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f01d400d950 tx=0x7f01d400de10 comp rx=0 tx=0).stop 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 shutdown_connections 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f01c803f5c0 0x7f01c8041a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01e419f7e0 0x7f01e419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01e4076f80 0x7f01e419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 --2- 172.21.15.103:0/653580328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01e4075b20 0x7f01e419e550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:29.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 >> 172.21.15.103:0/653580328 conn(0x7f01e40fe180 msgr2=0x7f01e4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:29.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 shutdown_connections 2024-09-11T00:13:29.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.254+0000 7f01e3200640 1 -- 172.21.15.103:0/653580328 wait complete. 2024-09-11T00:13:29.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:29 smithi103 ceph-mon[30578]: Reconfiguring mon.c (monmap changed)... 2024-09-11T00:13:29.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:29 smithi103 ceph-mon[30578]: Reconfiguring daemon mon.c on smithi183 2024-09-11T00:13:29.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:29 smithi103 ceph-mon[30578]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:29.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:29 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:29.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:29 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:29.409 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:29 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/653580328' entity='client.admin' 2024-09-11T00:13:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:29 smithi183 ceph-mon[35934]: Reconfiguring mon.c (monmap changed)... 2024-09-11T00:13:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:29 smithi183 ceph-mon[35934]: Reconfiguring daemon mon.c on smithi183 2024-09-11T00:13:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:29 smithi183 ceph-mon[35934]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:29 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:29 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:29 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/653580328' entity='client.admin' 2024-09-11T00:13:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:29 smithi117 ceph-mon[35525]: Reconfiguring mon.c (monmap changed)... 2024-09-11T00:13:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:29 smithi117 ceph-mon[35525]: Reconfiguring daemon mon.c on smithi183 2024-09-11T00:13:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:29 smithi117 ceph-mon[35525]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:29 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:29.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:29 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:29.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:29 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/653580328' entity='client.admin' 2024-09-11T00:13:30.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 -- 172.21.15.103:0/1302435528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 msgr2=0x7f5360077e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.542 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 --2- 172.21.15.103:0/1302435528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 0x7f5360077e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5354009860 tx=0x7f535402a340 comp rx=0 tx=0).stop 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 -- 172.21.15.103:0/1302435528 shutdown_connections 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 --2- 172.21.15.103:0/1302435528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360079530 0x7f5360079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 --2- 172.21.15.103:0/1302435528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5360078b70 0x7f5360078ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 --2- 172.21.15.103:0/1302435528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 0x7f5360077e10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 -- 172.21.15.103:0/1302435528 >> 172.21.15.103:0/1302435528 conn(0x7f5360100510 msgr2=0x7f5360102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 -- 172.21.15.103:0/1302435528 shutdown_connections 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.040+0000 7f5367800640 1 -- 172.21.15.103:0/1302435528 wait complete. 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 Processor -- start 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 -- start start 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 0x7f536019f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360078b70 0x7f536019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.543 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5360079530 0x7f53601a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53601149c0 con 0x7f5360079530 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5360114840 con 0x7f5360077a10 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.041+0000 7f5367800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5360114b40 con 0x7f5360078b70 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f535fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360078b70 0x7f536019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5364e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 0x7f536019f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5365800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5360079530 0x7f53601a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f535fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360078b70 0x7f536019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:38212/0 (socket says 172.21.15.103:38212) 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f535fe00640 1 -- 172.21.15.103:0/465241592 learned_addr learned my addr 172.21.15.103:0/465241592 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5364e00640 1 -- 172.21.15.103:0/465241592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360078b70 msgr2=0x7f536019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5364e00640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360078b70 0x7f536019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5364e00640 1 -- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5360079530 msgr2=0x7f53601a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.544 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5364e00640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5360079530 0x7f53601a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5364e00640 1 -- 172.21.15.103:0/465241592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53601a0ca0 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f535fe00640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360078b70 0x7f536019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.042+0000 7f5364e00640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 0x7f536019f170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f535402a860 tx=0x7f535402adb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.043+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5354037040 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.043+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f53540044b0 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.043+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53540035f0 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.043+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53601a0f30 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.043+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53601a74a0 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.044+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5360078ff0 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.044+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f535403a070 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.045+0000 7f535d600640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f534803f510 0x7f53480419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.045+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5354072f10 con 0x7f5360077a10 2024-09-11T00:13:30.545 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.045+0000 7f535fe00640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f534803f510 0x7f53480419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.046+0000 7f535fe00640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f534803f510 0x7f53480419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f534c0096c0 tx=0x7f534c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.050+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5354049020 con 0x7f5360077a10 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.179+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 7 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f5354033be0 con 0x7f5360077a10 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.257+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f5360063d50 con 0x7f5360077a10 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.264+0000 7f535d600640 1 -- 172.21.15.103:0/465241592 <== mon.2 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v15)=0 v15) ==== 116+0+0 (secure 0 0 0) 0x7f5354004050 con 0x7f5360077a10 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f534803f510 msgr2=0x7f53480419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f534803f510 0x7f53480419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f534c0096c0 tx=0x7f534c009260 comp rx=0 tx=0).stop 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 msgr2=0x7f536019f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 0x7f536019f170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f535402a860 tx=0x7f535402adb0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 shutdown_connections 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f534803f510 0x7f53480419d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5360079530 0x7f53601a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.546 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5360078b70 0x7f536019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 --2- 172.21.15.103:0/465241592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5360077a10 0x7f536019f170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 >> 172.21.15.103:0/465241592 conn(0x7f5360100510 msgr2=0x7f53601021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:30.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 shutdown_connections 2024-09-11T00:13:30.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.266+0000 7f5367800640 1 -- 172.21.15.103:0/465241592 wait complete. 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.827+0000 7f50ec400640 1 -- 172.21.15.103:0/2234311507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4103b90 msgr2=0x7f50e4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.827+0000 7f50ec400640 1 --2- 172.21.15.103:0/2234311507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4103b90 0x7f50e4104010 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f50d0009930 tx=0x7f50d002a420 comp rx=0 tx=0).stop 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.828+0000 7f50ec400640 1 -- 172.21.15.103:0/2234311507 shutdown_connections 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.828+0000 7f50ec400640 1 --2- 172.21.15.103:0/2234311507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e4104550 0x7f50e410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.828+0000 7f50ec400640 1 --2- 172.21.15.103:0/2234311507 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4103b90 0x7f50e4104010 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.828+0000 7f50ec400640 1 --2- 172.21.15.103:0/2234311507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e4102a30 0x7f50e4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.828+0000 7f50ec400640 1 -- 172.21.15.103:0/2234311507 >> 172.21.15.103:0/2234311507 conn(0x7f50e40fe320 msgr2=0x7f50e4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.828+0000 7f50ec400640 1 -- 172.21.15.103:0/2234311507 shutdown_connections 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.828+0000 7f50ec400640 1 -- 172.21.15.103:0/2234311507 wait complete. 2024-09-11T00:13:30.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.829+0000 7f50ec400640 1 Processor -- start 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.829+0000 7f50ec400640 1 -- start start 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50ec400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4102a30 0x7f50e4076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50ec400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e4103b90 0x7f50e4075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50ec400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e4104550 0x7f50e4075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50ec400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f50e410e3b0 con 0x7f50e4102a30 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50ec400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f50e410e230 con 0x7f50e4103b90 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50ec400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f50e410e530 con 0x7f50e4104550 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50e9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4102a30 0x7f50e4076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50e9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4102a30 0x7f50e4076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36536/0 (socket says 172.21.15.103:36536) 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50e9000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e4103b90 0x7f50e4075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50e9a00640 1 -- 172.21.15.103:0/1807253481 learned_addr learned my addr 172.21.15.103:0/1807253481 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50ea400640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e4104550 0x7f50e4075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.830+0000 7f50e9000640 1 -- 172.21.15.103:0/1807253481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e4104550 msgr2=0x7f50e4075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e9000640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e4104550 0x7f50e4075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e9000640 1 -- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4102a30 msgr2=0x7f50e4076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e9000640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4102a30 0x7f50e4076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e9000640 1 -- 172.21.15.103:0/1807253481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e4076340 con 0x7f50e4103b90 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e9a00640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4102a30 0x7f50e4076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e9000640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e4103b90 0x7f50e4075540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50d002a940 tx=0x7f50d00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e2000640 1 -- 172.21.15.103:0/1807253481 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d003f070 con 0x7f50e4103b90 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e2000640 1 -- 172.21.15.103:0/1807253481 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f50d0003a50 con 0x7f50e4103b90 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50e2000640 1 -- 172.21.15.103:0/1807253481 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0035030 con 0x7f50e4103b90 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e41a2c20 con 0x7f50e4103b90 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.831+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e41a3120 con 0x7f50e4103b90 2024-09-11T00:13:30.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.833+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e4102e30 con 0x7f50e4103b90 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.833+0000 7f50e2000640 1 -- 172.21.15.103:0/1807253481 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f50d00035f0 con 0x7f50e4103b90 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.834+0000 7f50e2000640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f50cc03f510 0x7f50cc0419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.834+0000 7f50e2000640 1 -- 172.21.15.103:0/1807253481 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f50d00746f0 con 0x7f50e4103b90 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.834+0000 7f50e9a00640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f50cc03f510 0x7f50cc0419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.835+0000 7f50e9a00640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f50cc03f510 0x7f50cc0419d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50d8011500 tx=0x7f50d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.839+0000 7f50e2000640 1 -- 172.21.15.103:0/1807253481 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f50d0030d90 con 0x7f50e4103b90 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.043+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f50e402e930 con 0x7f50e4103b90 2024-09-11T00:13:30.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.051+0000 7f50e2000640 1 -- 172.21.15.103:0/1807253481 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v13)=0 v13) ==== 125+0+0 (secure 0 0 0) 0x7f50d0031920 con 0x7f50e4103b90 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f50cc03f510 msgr2=0x7f50cc0419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f50cc03f510 0x7f50cc0419d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50d8011500 tx=0x7f50d8005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e4103b90 msgr2=0x7f50e4075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e4103b90 0x7f50e4075540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f50d002a940 tx=0x7f50d00045e0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 shutdown_connections 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f50cc03f510 0x7f50cc0419d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f50e4104550 0x7f50e4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e4103b90 0x7f50e4075540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 --2- 172.21.15.103:0/1807253481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f50e4102a30 0x7f50e4076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 >> 172.21.15.103:0/1807253481 conn(0x7f50e40fe320 msgr2=0x7f50e4107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 shutdown_connections 2024-09-11T00:13:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.054+0000 7f50ec400640 1 -- 172.21.15.103:0/1807253481 wait complete. 2024-09-11T00:13:30.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:30 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:30.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:30 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:30.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:30 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:30.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:30 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:30.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:30 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:30.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:30 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:30.858 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:30 smithi103 ceph-mon[30578]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:30 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:30 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:30 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:30 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:30 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:30 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:30.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:30 smithi117 ceph-mon[35525]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:30 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:30 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:30 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:30 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:30 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:30 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:30 smithi183 ceph-mon[35934]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 -- 172.21.15.103:0/4253075853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41063a0 msgr2=0x7f12e4106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 --2- 172.21.15.103:0/4253075853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41063a0 0x7f12e4106850 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009ca0 tx=0x7f12dc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.146 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 -- 172.21.15.103:0/4253075853 shutdown_connections 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 --2- 172.21.15.103:0/4253075853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41063a0 0x7f12e4106850 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 --2- 172.21.15.103:0/4253075853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e41037f0 0x7f12e4105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 --2- 172.21.15.103:0/4253075853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12e406b6e0 0x7f12e4103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 -- 172.21.15.103:0/4253075853 >> 172.21.15.103:0/4253075853 conn(0x7f12e40fd040 msgr2=0x7f12e40ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 -- 172.21.15.103:0/4253075853 shutdown_connections 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.939+0000 7f12eae00640 1 -- 172.21.15.103:0/4253075853 wait complete. 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 Processor -- start 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 -- start start 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12e406b6e0 0x7f12e4071b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41037f0 0x7f12e4072180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e41063a0 0x7f12e4072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:31.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4114860 con 0x7f12e41037f0 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41146e0 con 0x7f12e41063a0 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.941+0000 7f12eae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12e41149e0 con 0x7f12e406b6e0 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12ea400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e41063a0 0x7f12e4072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12ea400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e41063a0 0x7f12e4072dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:50056/0 (socket says 172.21.15.103:50056) 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12ea400640 1 -- 172.21.15.103:0/2440176265 learned_addr learned my addr 172.21.15.103:0/2440176265 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12e9a00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12e406b6e0 0x7f12e4071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12e9000640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41037f0 0x7f12e4072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12e9a00640 1 -- 172.21.15.103:0/2440176265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e41063a0 msgr2=0x7f12e4072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12e9a00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e41063a0 0x7f12e4072dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12e9a00640 1 -- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41037f0 msgr2=0x7f12e4072180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12e9a00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41037f0 0x7f12e4072180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.942+0000 7f12e9a00640 1 -- 172.21.15.103:0/2440176265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e4073660 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.943+0000 7f12e9a00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12e406b6e0 0x7f12e4071b30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f12d800e9f0 tx=0x7f12d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.943+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d800ccc0 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.943+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e4073950 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.943+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f12d8010890 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.943+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8018760 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.943+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e4074000 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.944+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12c40050b0 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.945+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f12d8018900 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.945+0000 7f12e2000640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12cc03f5c0 0x7f12cc041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.945+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f12d8054120 con 0x7f12e406b6e0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.946+0000 7f12e9000640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12cc03f5c0 0x7f12cc041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.946+0000 7f12e9000640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12cc03f5c0 0x7f12cc041a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12d00095f0 tx=0x7f12d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:31.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:29.950+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f12d80071c0 con 0x7f12e406b6e0 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.172+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f12c40053a0 con 0x7f12e406b6e0 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.179+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 7 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f12d8025da0 con 0x7f12e406b6e0 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.179+0000 7f12e2000640 1 -- 172.21.15.103:0/2440176265 <== mon.1 v2:172.21.15.183:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v14)=0 v14) ==== 114+0+0 (secure 0 0 0) 0x7f12d8022980 con 0x7f12e406b6e0 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12cc03f5c0 msgr2=0x7f12cc041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12cc03f5c0 0x7f12cc041a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f12d00095f0 tx=0x7f12d0005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12e406b6e0 msgr2=0x7f12e4071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12e406b6e0 0x7f12e4071b30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f12d800e9f0 tx=0x7f12d800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 shutdown_connections 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12cc03f5c0 0x7f12cc041a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12e41063a0 0x7f12e4072dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12e41037f0 0x7f12e4072180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 --2- 172.21.15.103:0/2440176265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12e406b6e0 0x7f12e4071b30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 >> 172.21.15.103:0/2440176265 conn(0x7f12e40fd040 msgr2=0x7f12e4101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:31.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 shutdown_connections 2024-09-11T00:13:31.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:30.182+0000 7f12eae00640 1 -- 172.21.15.103:0/2440176265 wait complete. 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.611+0000 7fb2ae400640 1 -- 172.21.15.103:0/2795712974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a8105a00 msgr2=0x7fb2a8107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.611+0000 7fb2ae400640 1 --2- 172.21.15.103:0/2795712974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a8105a00 0x7fb2a8107e70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb294009900 tx=0x7fb29402a410 comp rx=0 tx=0).stop 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.612+0000 7fb2ae400640 1 -- 172.21.15.103:0/2795712974 shutdown_connections 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.612+0000 7fb2ae400640 1 --2- 172.21.15.103:0/2795712974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a81085b0 0x7fb2a810aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.612+0000 7fb2ae400640 1 --2- 172.21.15.103:0/2795712974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a8105a00 0x7fb2a8107e70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.612+0000 7fb2ae400640 1 --2- 172.21.15.103:0/2795712974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2a806b6e0 0x7fb2a8105470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.612+0000 7fb2ae400640 1 -- 172.21.15.103:0/2795712974 >> 172.21.15.103:0/2795712974 conn(0x7fb2a80fd040 msgr2=0x7fb2a80ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.612+0000 7fb2ae400640 1 -- 172.21.15.103:0/2795712974 shutdown_connections 2024-09-11T00:13:32.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.612+0000 7fb2ae400640 1 -- 172.21.15.103:0/2795712974 wait complete. 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 Processor -- start 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 -- start start 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a806b6e0 0x7fb2a8102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a8105a00 0x7fb2a8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2a81085b0 0x7fb2a8101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a810e2d0 con 0x7fb2a806b6e0 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a810e150 con 0x7fb2a8105a00 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ae400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a810e450 con 0x7fb2a81085b0 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.613+0000 7fb2ad000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a806b6e0 0x7fb2a8102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ad000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a806b6e0 0x7fb2a8102ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33492/0 (socket says 172.21.15.103:33492) 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ad000640 1 -- 172.21.15.103:0/4236757407 learned_addr learned my addr 172.21.15.103:0/4236757407 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ada00640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2a81085b0 0x7fb2a8101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2a7e00640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a8105a00 0x7fb2a8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ada00640 1 -- 172.21.15.103:0/4236757407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a8105a00 msgr2=0x7fb2a8101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ada00640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a8105a00 0x7fb2a8101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ada00640 1 -- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a806b6e0 msgr2=0x7fb2a8102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ada00640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a806b6e0 0x7fb2a8102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ada00640 1 -- 172.21.15.103:0/4236757407 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a8102210 con 0x7fb2a81085b0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.614+0000 7fb2ada00640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2a81085b0 0x7fb2a8101950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000ca30 tx=0x7fb2a000cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.615+0000 7fb2a5600640 1 -- 172.21.15.103:0/4236757407 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0007d90 con 0x7fb2a81085b0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.615+0000 7fb2a5600640 1 -- 172.21.15.103:0/4236757407 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fb2a0005020 con 0x7fb2a81085b0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.615+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a81a2b40 con 0x7fb2a81085b0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.615+0000 7fb2a5600640 1 -- 172.21.15.103:0/4236757407 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000f6b0 con 0x7fb2a81085b0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.615+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a81a3050 con 0x7fb2a81085b0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.616+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a8105e60 con 0x7fb2a81085b0 2024-09-11T00:13:32.793 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.616+0000 7fb2a5600640 1 -- 172.21.15.103:0/4236757407 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7fb2a0004a40 con 0x7fb2a81085b0 2024-09-11T00:13:32.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.617+0000 7fb2a5600640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb29003f5c0 0x7fb290041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.617+0000 7fb2a5600640 1 -- 172.21.15.103:0/4236757407 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb2a0054090 con 0x7fb2a81085b0 2024-09-11T00:13:32.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.617+0000 7fb2ad000640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb29003f5c0 0x7fb290041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.618+0000 7fb2ad000640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb29003f5c0 0x7fb290041a80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009cd0 tx=0x7fb29c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:32.794 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.622+0000 7fb2a5600640 1 -- 172.21.15.103:0/4236757407 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb2a0014070 con 0x7fb2a81085b0 2024-09-11T00:13:32.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.820+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fb2a802e930 con 0x7fb2a81085b0 2024-09-11T00:13:32.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.827+0000 7fb2a5600640 1 -- 172.21.15.103:0/4236757407 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v17)=0 v17) ==== 117+0+0 (secure 0 0 0) 0x7fb2a000f850 con 0x7fb2a81085b0 2024-09-11T00:13:32.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb29003f5c0 msgr2=0x7fb290041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb29003f5c0 0x7fb290041a80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009cd0 tx=0x7fb29c005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:32.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2a81085b0 msgr2=0x7fb2a8101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2a81085b0 0x7fb2a8101950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000ca30 tx=0x7fb2a000cef0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 shutdown_connections 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb29003f5c0 0x7fb290041a80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb2a81085b0 0x7fb2a8101950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2a8105a00 0x7fb2a8101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 --2- 172.21.15.103:0/4236757407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a806b6e0 0x7fb2a8102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.829+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 >> 172.21.15.103:0/4236757407 conn(0x7fb2a80fd040 msgr2=0x7fb2a8103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.830+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 shutdown_connections 2024-09-11T00:13:32.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.830+0000 7fb2ae400640 1 -- 172.21.15.103:0/4236757407 wait complete. 2024-09-11T00:13:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 -- 172.21.15.103:0/1901138643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b10102890 msgr2=0x7f1b10102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 --2- 172.21.15.103:0/1901138643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b10102890 0x7f1b10102c90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009930 tx=0x7f1b0402a420 comp rx=0 tx=0).stop 2024-09-11T00:13:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 -- 172.21.15.103:0/1901138643 shutdown_connections 2024-09-11T00:13:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 --2- 172.21.15.103:0/1901138643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b101043b0 0x7f1b1010ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 --2- 172.21.15.103:0/1901138643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b101039f0 0x7f1b10103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 --2- 172.21.15.103:0/1901138643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b10102890 0x7f1b10102c90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 -- 172.21.15.103:0/1901138643 >> 172.21.15.103:0/1901138643 conn(0x7f1b100fe180 msgr2=0x7f1b101005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 -- 172.21.15.103:0/1901138643 shutdown_connections 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.467+0000 7f1b17000640 1 -- 172.21.15.103:0/1901138643 wait complete. 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 Processor -- start 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 -- start start 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b10102890 0x7f1b1019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b101039f0 0x7f1b1019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b101043b0 0x7f1b1019be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1010e2e0 con 0x7f1b101039f0 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1010e160 con 0x7f1b101043b0 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.468+0000 7f1b17000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1010e460 con 0x7f1b10102890 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b101039f0 0x7f1b1019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b101039f0 0x7f1b1019b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33478/0 (socket says 172.21.15.103:33478) 2024-09-11T00:13:32.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 -- 172.21.15.103:0/746919119 learned_addr learned my addr 172.21.15.103:0/746919119 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b16600640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b101043b0 0x7f1b1019be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15c00640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b10102890 0x7f1b1019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 -- 172.21.15.103:0/746919119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b10102890 msgr2=0x7f1b1019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b10102890 0x7f1b1019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 -- 172.21.15.103:0/746919119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b101043b0 msgr2=0x7f1b1019be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b101043b0 0x7f1b1019be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 -- 172.21.15.103:0/746919119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1019c6f0 con 0x7f1b101039f0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.469+0000 7f1b15200640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b101039f0 0x7f1b1019b210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00e9f0 tx=0x7f1afc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.470+0000 7f1b0e000640 1 -- 172.21.15.103:0/746919119 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc00cd80 con 0x7f1b101039f0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.470+0000 7f1b0e000640 1 -- 172.21.15.103:0/746919119 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f1afc0108a0 con 0x7f1b101039f0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.470+0000 7f1b0e000640 1 -- 172.21.15.103:0/746919119 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc018760 con 0x7f1b101039f0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.470+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1019c9e0 con 0x7f1b101039f0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.470+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b101a30c0 con 0x7f1b101039f0 2024-09-11T00:13:32.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.471+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b10102c90 con 0x7f1b101039f0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.471+0000 7f1b0e000640 1 -- 172.21.15.103:0/746919119 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f1afc010430 con 0x7f1b101039f0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.472+0000 7f1b0e000640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1af803f5c0 0x7f1af8041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.472+0000 7f1b0e000640 1 -- 172.21.15.103:0/746919119 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1afc054180 con 0x7f1b101039f0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.472+0000 7f1b15c00640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1af803f5c0 0x7f1af8041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.473+0000 7f1b15c00640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1af803f5c0 0x7f1af8041a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b04005ab0 tx=0x7f1b04005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.476+0000 7f1b0e000640 1 -- 172.21.15.103:0/746919119 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1afc018e20 con 0x7f1b101039f0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.686+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f1b101a3290 con 0x7f1b101039f0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.691+0000 7f1b0e000640 1 -- 172.21.15.103:0/746919119 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f1afc01f310 con 0x7f1b101039f0 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1af803f5c0 msgr2=0x7f1af8041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1af803f5c0 0x7f1af8041a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b04005ab0 tx=0x7f1b04005a40 comp rx=0 tx=0).stop 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b101039f0 msgr2=0x7f1b1019b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b101039f0 0x7f1b1019b210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00e9f0 tx=0x7f1afc00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 shutdown_connections 2024-09-11T00:13:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1af803f5c0 0x7f1af8041a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b101043b0 0x7f1b1019be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b101039f0 0x7f1b1019b210 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 --2- 172.21.15.103:0/746919119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b10102890 0x7f1b1019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 >> 172.21.15.103:0/746919119 conn(0x7f1b100fe180 msgr2=0x7f1b10109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.694+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 shutdown_connections 2024-09-11T00:13:32.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:32.695+0000 7f1b17000640 1 -- 172.21.15.103:0/746919119 wait complete. 2024-09-11T00:13:33.584 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:33 smithi103 ceph-mon[30578]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:33.584 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:33 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/746919119' entity='client.admin' 2024-09-11T00:13:33.584 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:33 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:33 smithi117 ceph-mon[35525]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:33 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/746919119' entity='client.admin' 2024-09-11T00:13:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:33 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:33.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:33 smithi183 ceph-mon[35934]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:33.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:33 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/746919119' entity='client.admin' 2024-09-11T00:13:33.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:33 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:33.848 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:33.859 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:33.868 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:34.425 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:34.425 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:34.469 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi103') 2024-09-11T00:13:34.787 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 -- 172.21.15.103:0/3209548005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4880075ac0 msgr2=0x7f4880075f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 --2- 172.21.15.103:0/3209548005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4880075ac0 0x7f4880075f40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f486c009900 tx=0x7f486c02a410 comp rx=0 tx=0).stop 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 -- 172.21.15.103:0/3209548005 shutdown_connections 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 --2- 172.21.15.103:0/3209548005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f488010a810 0x7f488010cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 --2- 172.21.15.103:0/3209548005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4880075ac0 0x7f4880075f40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 --2- 172.21.15.103:0/3209548005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4880076f20 0x7f4880075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 -- 172.21.15.103:0/3209548005 >> 172.21.15.103:0/3209548005 conn(0x7f48800fe150 msgr2=0x7f4880100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 -- 172.21.15.103:0/3209548005 shutdown_connections 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.582+0000 7f4885a00640 1 -- 172.21.15.103:0/3209548005 wait complete. 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.583+0000 7f4885a00640 1 Processor -- start 2024-09-11T00:13:34.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885a00640 1 -- start start 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4880075ac0 0x7f488019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880076f20 0x7f488019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f488010a810 0x7f488019be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f48801104a0 con 0x7f488010a810 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4880110320 con 0x7f4880076f20 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4880110620 con 0x7f4880075ac0 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f488010a810 0x7f488019be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f488010a810 0x7f488019be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33514/0 (socket says 172.21.15.103:33514) 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f4885000640 1 -- 172.21.15.103:0/277484660 learned_addr learned my addr 172.21.15.103:0/277484660 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f487fe00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4880075ac0 0x7f488019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.584+0000 7f487f400640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880076f20 0x7f488019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f4885000640 1 -- 172.21.15.103:0/277484660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4880075ac0 msgr2=0x7f488019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:34.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f4885000640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4880075ac0 0x7f488019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f4885000640 1 -- 172.21.15.103:0/277484660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880076f20 msgr2=0x7f488019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f4885000640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880076f20 0x7f488019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f4885000640 1 -- 172.21.15.103:0/277484660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488019c720 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f487f400640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880076f20 0x7f488019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f4885000640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f488010a810 0x7f488019be80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f487800eb40 tx=0x7f487800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878004b20 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f4878004cc0 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878010690 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.585+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488019ca10 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.586+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801a30f0 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.587+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4880110c60 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.587+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f4878005020 con 0x7f488010a810 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.587+0000 7f487cc00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f486403f5c0 0x7f4864041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:34.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.587+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f48780544f0 con 0x7f488010a810 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.588+0000 7f487fe00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f486403f5c0 0x7f4864041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.588+0000 7f487fe00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f486403f5c0 0x7f4864041a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4870009bd0 tx=0x7f4870005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.592+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4878011dc0 con 0x7f488010a810 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.780+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f48801a3490 con 0x7f488010a810 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.785+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v18)=0 v18) ==== 123+0+0 (secure 0 0 0) 0x7f4878007d30 con 0x7f488010a810 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.786+0000 7f487cc00640 1 -- 172.21.15.103:0/277484660 <== mon.0 v2:172.21.15.103:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f487801faf0 con 0x7f488010a810 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f486403f5c0 msgr2=0x7f4864041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f486403f5c0 0x7f4864041a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4870009bd0 tx=0x7f4870005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f488010a810 msgr2=0x7f488019be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f488010a810 0x7f488019be80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f487800eb40 tx=0x7f487800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 shutdown_connections 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f486403f5c0 0x7f4864041a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f488010a810 0x7f488019be80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4880076f20 0x7f488019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 --2- 172.21.15.103:0/277484660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4880075ac0 0x7f488019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:34.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 >> 172.21.15.103:0/277484660 conn(0x7f48800fe150 msgr2=0x7f48800fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:34.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 shutdown_connections 2024-09-11T00:13:34.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:33.788+0000 7f4885a00640 1 -- 172.21.15.103:0/277484660 wait complete. 2024-09-11T00:13:34.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/277484660' entity='client.admin' 2024-09-11T00:13:34.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:34.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:34.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:34.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:34.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:13:34.929 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:34.930 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:34.930 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: Deploying daemon mgr.z on smithi183 2024-09-11T00:13:34.930 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:34 smithi183 ceph-mon[35934]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:34.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/277484660' entity='client.admin' 2024-09-11T00:13:34.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:34.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:34.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:34.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:34.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:13:34.980 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:34.981 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:34.981 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: Deploying daemon mgr.z on smithi183 2024-09-11T00:13:34.981 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:34 smithi103 ceph-mon[30578]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:35.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/277484660' entity='client.admin' 2024-09-11T00:13:35.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:35.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:35.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:35.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:35.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:13:35.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:35.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:35.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: Deploying daemon mgr.z on smithi183 2024-09-11T00:13:35.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:34 smithi117 ceph-mon[35525]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:35.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 -- 172.21.15.103:0/3408440679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae0105a00 msgr2=0x7fdae0107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 --2- 172.21.15.103:0/3408440679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae0105a00 0x7fdae0107e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009860 tx=0x7fdacc02a340 comp rx=0 tx=0).stop 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 -- 172.21.15.103:0/3408440679 shutdown_connections 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 --2- 172.21.15.103:0/3408440679 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae01085b0 0x7fdae010aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 --2- 172.21.15.103:0/3408440679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae0105a00 0x7fdae0107e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 --2- 172.21.15.103:0/3408440679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae006b6e0 0x7fdae0105470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 -- 172.21.15.103:0/3408440679 >> 172.21.15.103:0/3408440679 conn(0x7fdae00fd040 msgr2=0x7fdae00ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.572+0000 7fdadf200640 1 -- 172.21.15.103:0/3408440679 shutdown_connections 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.573+0000 7fdadf200640 1 -- 172.21.15.103:0/3408440679 wait complete. 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.573+0000 7fdadf200640 1 Processor -- start 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadf200640 1 -- start start 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae006b6e0 0x7fdae019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadf200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae0105a00 0x7fdae019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae01085b0 0x7fdae019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdae010e3c0 con 0x7fdae006b6e0 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdae010e240 con 0x7fdae01085b0 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadf200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdae010e540 con 0x7fdae0105a00 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae006b6e0 0x7fdae019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae0105a00 0x7fdae019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae006b6e0 0x7fdae019acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33542/0 (socket says 172.21.15.103:33542) 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadde00640 1 -- 172.21.15.103:0/2516070993 learned_addr learned my addr 172.21.15.103:0/2516070993 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadd400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae0105a00 0x7fdae019b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:53040/0 (socket says 172.21.15.103:53040) 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdade800640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae01085b0 0x7fdae019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadd400640 1 -- 172.21.15.103:0/2516070993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae01085b0 msgr2=0x7fdae019bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadd400640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae01085b0 0x7fdae019bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadd400640 1 -- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae006b6e0 msgr2=0x7fdae019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadd400640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae006b6e0 0x7fdae019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.574+0000 7fdadd400640 1 -- 172.21.15.103:0/2516070993 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae019c7e0 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.575+0000 7fdadd400640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae0105a00 0x7fdae019b300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009780 tx=0x7fdacc030980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.575+0000 7fdad6a00640 1 -- 172.21.15.103:0/2516070993 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc037040 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.575+0000 7fdad6a00640 1 -- 172.21.15.103:0/2516070993 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdacc033e50 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.575+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae019ca70 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.575+0000 7fdad6a00640 1 -- 172.21.15.103:0/2516070993 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc005320 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.575+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae01a2fe0 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.576+0000 7fdad6a00640 1 -- 172.21.15.103:0/2516070993 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7fdacc03a070 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.576+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae01a31b0 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.577+0000 7fdad6a00640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdac403f5c0 0x7fdac4041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.577+0000 7fdad6a00640 1 -- 172.21.15.103:0/2516070993 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdacc073190 con 0x7fdae0105a00 2024-09-11T00:13:35.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.577+0000 7fdadde00640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdac403f5c0 0x7fdac4041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:35.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.578+0000 7fdadde00640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdac403f5c0 0x7fdac4041a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdad0005c80 tx=0x7fdad0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:35.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.582+0000 7fdad6a00640 1 -- 172.21.15.103:0/2516070993 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdacc043360 con 0x7fdae0105a00 2024-09-11T00:13:35.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.780+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fdae01a3380 con 0x7fdae0105a00 2024-09-11T00:13:35.786 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.785+0000 7fdad6a00640 1 -- 172.21.15.103:0/2516070993 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fdacc0371e0 con 0x7fdae0105a00 2024-09-11T00:13:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdac403f5c0 msgr2=0x7fdac4041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdac403f5c0 0x7fdac4041a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdad0005c80 tx=0x7fdad0009260 comp rx=0 tx=0).stop 2024-09-11T00:13:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae0105a00 msgr2=0x7fdae019b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae0105a00 0x7fdae019b300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009780 tx=0x7fdacc030980 comp rx=0 tx=0).stop 2024-09-11T00:13:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 shutdown_connections 2024-09-11T00:13:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdac403f5c0 0x7fdac4041a80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.789 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdae01085b0 0x7fdae019bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdae0105a00 0x7fdae019b300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 --2- 172.21.15.103:0/2516070993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdae006b6e0 0x7fdae019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:35.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 >> 172.21.15.103:0/2516070993 conn(0x7fdae00fd040 msgr2=0x7fdae00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:35.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 shutdown_connections 2024-09-11T00:13:35.790 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.787+0000 7fdadf200640 1 -- 172.21.15.103:0/2516070993 wait complete. 2024-09-11T00:13:36.190 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:35 smithi183 systemd[1]: Starting Ceph mgr.z for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 -- 172.21.15.103:0/1411144269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74106780 msgr2=0x7eff7410d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 --2- 172.21.15.103:0/1411144269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74106780 0x7eff7410d100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009cd0 tx=0x7eff6c02a8f0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 -- 172.21.15.103:0/1411144269 shutdown_connections 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 --2- 172.21.15.103:0/1411144269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74106780 0x7eff7410d100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 --2- 172.21.15.103:0/1411144269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff74105dc0 0x7eff74106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 --2- 172.21.15.103:0/1411144269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74104c60 0x7eff74105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 -- 172.21.15.103:0/1411144269 >> 172.21.15.103:0/1411144269 conn(0x7eff74100510 msgr2=0x7eff74102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 -- 172.21.15.103:0/1411144269 shutdown_connections 2024-09-11T00:13:36.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.269+0000 7eff79c00640 1 -- 172.21.15.103:0/1411144269 wait complete. 2024-09-11T00:13:36.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.270+0000 7eff79c00640 1 Processor -- start 2024-09-11T00:13:36.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.270+0000 7eff79c00640 1 -- start start 2024-09-11T00:13:36.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.270+0000 7eff79c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff74104c60 0x7eff7419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.270+0000 7eff79c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74105dc0 0x7eff7419b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff79c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74106780 0x7eff7419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff79c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7eff741107f0 con 0x7eff74105dc0 2024-09-11T00:13:36.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff79c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7eff74110670 con 0x7eff74104c60 2024-09-11T00:13:36.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff79c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7eff74110970 con 0x7eff74106780 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74106780 0x7eff7419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff72a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff74104c60 0x7eff7419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74106780 0x7eff7419c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:53060/0 (socket says 172.21.15.103:53060) 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 -- 172.21.15.103:0/3141650742 learned_addr learned my addr 172.21.15.103:0/3141650742 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff72000640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74105dc0 0x7eff7419b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 -- 172.21.15.103:0/3141650742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff74104c60 msgr2=0x7eff7419aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff74104c60 0x7eff7419aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 -- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74105dc0 msgr2=0x7eff7419b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74105dc0 0x7eff7419b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.271+0000 7eff73400640 1 -- 172.21.15.103:0/3141650742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7419ca10 con 0x7eff74106780 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.272+0000 7eff72a00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff74104c60 0x7eff7419aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.272+0000 7eff73400640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74106780 0x7eff7419c170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff6c030820 tx=0x7eff6c003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.272+0000 7eff6b400640 1 -- 172.21.15.103:0/3141650742 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c004240 con 0x7eff74106780 2024-09-11T00:13:36.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.272+0000 7eff6b400640 1 -- 172.21.15.103:0/3141650742 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff6c040070 con 0x7eff74106780 2024-09-11T00:13:36.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.272+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7419cca0 con 0x7eff74106780 2024-09-11T00:13:36.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.272+0000 7eff6b400640 1 -- 172.21.15.103:0/3141650742 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6c045b50 con 0x7eff74106780 2024-09-11T00:13:36.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.272+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff741a33e0 con 0x7eff74106780 2024-09-11T00:13:36.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.274+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff7410bf10 con 0x7eff74106780 2024-09-11T00:13:36.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.274+0000 7eff6b400640 1 -- 172.21.15.103:0/3141650742 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7eff6c04d070 con 0x7eff74106780 2024-09-11T00:13:36.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.274+0000 7eff6b400640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7eff5803f510 0x7eff580419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.274+0000 7eff6b400640 1 -- 172.21.15.103:0/3141650742 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7eff6c075210 con 0x7eff74106780 2024-09-11T00:13:36.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.275+0000 7eff72a00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7eff5803f510 0x7eff580419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.275+0000 7eff72a00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7eff5803f510 0x7eff580419d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff64009fc0 tx=0x7eff64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:36.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.279+0000 7eff6b400640 1 -- 172.21.15.103:0/3141650742 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7eff6c043370 con 0x7eff74106780 2024-09-11T00:13:36.456 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:36 smithi183 podman[37495]: 2024-09-11 00:13:36.09217365 +0000 UTC m=+0.015480804 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:13:36.456 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:36 smithi183 podman[37495]: 2024-09-11 00:13:36.224415365 +0000 UTC m=+0.147722512 container create 81633643c467e3ed145038fd07fac50dacfae5118462b3f9931073d7eb959905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T00:13:36.456 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:36 smithi183 podman[37495]: 2024-09-11 00:13:36.426551949 +0000 UTC m=+0.349859096 container init 81633643c467e3ed145038fd07fac50dacfae5118462b3f9931073d7eb959905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T00:13:36.456 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:36 smithi183 podman[37495]: 2024-09-11 00:13:36.429591091 +0000 UTC m=+0.352898238 container start 81633643c467e3ed145038fd07fac50dacfae5118462b3f9931073d7eb959905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T00:13:36.456 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mgr[37515]: -- 172.21.15.183:0/3928240539 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fefc1fd0e0 con 0x55fefc1db000 2024-09-11T00:13:36.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.483+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7eff7402e5c0 con 0x7eff74106780 2024-09-11T00:13:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.489+0000 7eff6b400640 1 -- 172.21.15.103:0/3141650742 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v21)=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7eff6c045470 con 0x7eff74106780 2024-09-11T00:13:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7eff5803f510 msgr2=0x7eff580419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7eff5803f510 0x7eff580419d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7eff64009fc0 tx=0x7eff64005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74106780 msgr2=0x7eff7419c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74106780 0x7eff7419c170 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff6c030820 tx=0x7eff6c003ba0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 shutdown_connections 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7eff5803f510 0x7eff580419d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7eff74106780 0x7eff7419c170 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7eff74105dc0 0x7eff7419b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 --2- 172.21.15.103:0/3141650742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff74104c60 0x7eff7419aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 >> 172.21.15.103:0/3141650742 conn(0x7eff74100510 msgr2=0x7eff741090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 shutdown_connections 2024-09-11T00:13:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.492+0000 7eff79c00640 1 -- 172.21.15.103:0/3141650742 wait complete. 2024-09-11T00:13:36.780 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:36 smithi183 bash[37495]: 81633643c467e3ed145038fd07fac50dacfae5118462b3f9931073d7eb959905 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:13:36 smithi183 systemd[1]: Started Ceph mgr.z for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1650119712' entity='client.admin' 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:13:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:36.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:36.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 -- 172.21.15.103:0/2045180229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0075b20 msgr2=0x7f81b0075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 --2- 172.21.15.103:0/2045180229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0075b20 0x7f81b0075fa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f819c009900 tx=0x7f819c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 -- 172.21.15.103:0/2045180229 shutdown_connections 2024-09-11T00:13:36.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 --2- 172.21.15.103:0/2045180229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81b01063b0 0x7f81b0111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 --2- 172.21.15.103:0/2045180229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0075b20 0x7f81b0075fa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 --2- 172.21.15.103:0/2045180229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0076f80 0x7f81b00755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 -- 172.21.15.103:0/2045180229 >> 172.21.15.103:0/2045180229 conn(0x7f81b00fe180 msgr2=0x7f81b01005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 -- 172.21.15.103:0/2045180229 shutdown_connections 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.534+0000 7f81af200640 1 -- 172.21.15.103:0/2045180229 wait complete. 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.535+0000 7f81af200640 1 Processor -- start 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.535+0000 7f81af200640 1 -- start start 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81af200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0075b20 0x7f81b019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81af200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0076f80 0x7f81b019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81af200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81b01a0310 0x7f81b01a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81af200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0114a00 con 0x7f81b0076f80 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81af200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0114880 con 0x7f81b0075b20 2024-09-11T00:13:36.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81af200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81b0114b80 con 0x7f81b01a0310 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81ad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0076f80 0x7f81b019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81ad400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0076f80 0x7f81b019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33574/0 (socket says 172.21.15.103:33574) 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81ad400640 1 -- 172.21.15.103:0/1317783751 learned_addr learned my addr 172.21.15.103:0/1317783751 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81ade00640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0075b20 0x7f81b019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.536+0000 7f81ae800640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81b01a0310 0x7f81b01a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81ad400640 1 -- 172.21.15.103:0/1317783751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81b01a0310 msgr2=0x7f81b01a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81ad400640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81b01a0310 0x7f81b01a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81ad400640 1 -- 172.21.15.103:0/1317783751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0075b20 msgr2=0x7f81b019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81ad400640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0075b20 0x7f81b019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81ad400640 1 -- 172.21.15.103:0/1317783751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b01a0f90 con 0x7f81b0076f80 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81ade00640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0075b20 0x7f81b019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:36.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81ad400640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0076f80 0x7f81b019f6d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f819c004770 tx=0x7f819c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81a6a00640 1 -- 172.21.15.103:0/1317783751 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c03f070 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81a6a00640 1 -- 172.21.15.103:0/1317783751 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f819c03b460 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a73b0 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81a6a00640 1 -- 172.21.15.103:0/1317783751 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c035070 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.537+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a77a0 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.539+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b0075fa0 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.539+0000 7f81a6a00640 1 -- 172.21.15.103:0/1317783751 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f819c04b060 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.540+0000 7f81a6a00640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f819403f5c0 0x7f8194041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.540+0000 7f81a6a00640 1 -- 172.21.15.103:0/1317783751 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f819c073fd0 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.540+0000 7f81ade00640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f819403f5c0 0x7f8194041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.541+0000 7f81ade00640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f819403f5c0 0x7f8194041a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81a00058c0 tx=0x7f81a0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.545+0000 7f81a6a00640 1 -- 172.21.15.103:0/1317783751 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f819c0303a0 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.742+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f81b0105180 con 0x7f81b0076f80 2024-09-11T00:13:36.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.745+0000 7f81a6a00640 1 -- 172.21.15.103:0/1317783751 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v22)=0 v22) ==== 119+0+0 (secure 0 0 0) 0x7f819c041de0 con 0x7f81b0076f80 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f819403f5c0 msgr2=0x7f8194041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f819403f5c0 0x7f8194041a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81a00058c0 tx=0x7f81a0005850 comp rx=0 tx=0).stop 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0076f80 msgr2=0x7f81b019f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0076f80 0x7f81b019f6d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f819c004770 tx=0x7f819c0042f0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 shutdown_connections 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f819403f5c0 0x7f8194041a80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81b01a0310 0x7f81b01a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81b0076f80 0x7f81b019f6d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 --2- 172.21.15.103:0/1317783751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0075b20 0x7f81b019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 >> 172.21.15.103:0/1317783751 conn(0x7f81b00fe180 msgr2=0x7f81b0102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.747+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 shutdown_connections 2024-09-11T00:13:36.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:36.748+0000 7f81af200640 1 -- 172.21.15.103:0/1317783751 wait complete. 2024-09-11T00:13:37.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1650119712' entity='client.admin' 2024-09-11T00:13:37.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:37.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:37.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:37.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:37.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:37.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:37.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:37.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:13:37.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:37.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1650119712' entity='client.admin' 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-11T00:13:37.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:37.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:37 smithi103 ceph-mon[30578]: Deploying daemon mgr.y on smithi117 2024-09-11T00:13:37.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:37 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1317783751' entity='client.admin' 2024-09-11T00:13:37.859 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:37 smithi183 ceph-mon[35934]: Deploying daemon mgr.y on smithi117 2024-09-11T00:13:37.859 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:37 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1317783751' entity='client.admin' 2024-09-11T00:13:37.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:37 smithi117 ceph-mon[35525]: Deploying daemon mgr.y on smithi117 2024-09-11T00:13:37.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:37 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1317783751' entity='client.admin' 2024-09-11T00:13:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 -- 172.21.15.103:0/458719741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 msgr2=0x7f27fc10aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 --2- 172.21.15.103:0/458719741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 0x7f27fc10aec0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009cd0 tx=0x7f27f402a8e0 comp rx=0 tx=0).stop 2024-09-11T00:13:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 -- 172.21.15.103:0/458719741 shutdown_connections 2024-09-11T00:13:37.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 --2- 172.21.15.103:0/458719741 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 0x7f27fc10aec0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 --2- 172.21.15.103:0/458719741 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc103b80 0x7f27fc104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 --2- 172.21.15.103:0/458719741 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc102a20 0x7f27fc102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 -- 172.21.15.103:0/458719741 >> 172.21.15.103:0/458719741 conn(0x7f27fc0fe2f0 msgr2=0x7f27fc100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 -- 172.21.15.103:0/458719741 shutdown_connections 2024-09-11T00:13:37.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.964+0000 7f2802600640 1 -- 172.21.15.103:0/458719741 wait complete. 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.965+0000 7f2802600640 1 Processor -- start 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.965+0000 7f2802600640 1 -- start start 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f2802600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc102a20 0x7f27fc076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f2802600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc103b80 0x7f27fc075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f2802600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 0x7f27fc075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f2802600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10e5b0 con 0x7f27fc104540 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f2802600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10e430 con 0x7f27fc102a20 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f2802600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc10e730 con 0x7f27fc103b80 2024-09-11T00:13:37.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f27faa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc103b80 0x7f27fc075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:37.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f27fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc102a20 0x7f27fc076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:37.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f27faa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc103b80 0x7f27fc075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:53094/0 (socket says 172.21.15.103:53094) 2024-09-11T00:13:37.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f27fb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc102a20 0x7f27fc076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:39910/0 (socket says 172.21.15.103:39910) 2024-09-11T00:13:37.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f27faa00640 1 -- 172.21.15.103:0/1922168153 learned_addr learned my addr 172.21.15.103:0/1922168153 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:37.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.966+0000 7f27fbe00640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 0x7f27fc075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:37.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27faa00640 1 -- 172.21.15.103:0/1922168153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc102a20 msgr2=0x7f27fc076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:37.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27faa00640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc102a20 0x7f27fc076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:37.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27faa00640 1 -- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 msgr2=0x7f27fc075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:37.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27faa00640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 0x7f27fc075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:37.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27faa00640 1 -- 172.21.15.103:0/1922168153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc076340 con 0x7f27fc103b80 2024-09-11T00:13:37.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27fb400640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc102a20 0x7f27fc076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:37.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27faa00640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc103b80 0x7f27fc075540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27e400e9f0 tx=0x7f27e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:37.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27f3e00640 1 -- 172.21.15.103:0/1922168153 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e400cd80 con 0x7f27fc103b80 2024-09-11T00:13:37.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc1a2dc0 con 0x7f27fc103b80 2024-09-11T00:13:37.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27f3e00640 1 -- 172.21.15.103:0/1922168153 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f27e40108c0 con 0x7f27fc103b80 2024-09-11T00:13:37.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f27f3e00640 1 -- 172.21.15.103:0/1922168153 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e40187d0 con 0x7f27fc103b80 2024-09-11T00:13:37.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.967+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc1a3280 con 0x7f27fc103b80 2024-09-11T00:13:37.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.969+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27fc109da0 con 0x7f27fc103b80 2024-09-11T00:13:37.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.969+0000 7f27f3e00640 1 -- 172.21.15.103:0/1922168153 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f27e4010430 con 0x7f27fc103b80 2024-09-11T00:13:37.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.969+0000 7f27f3e00640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27e003f510 0x7f27e00419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:37.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.970+0000 7f27f3e00640 1 -- 172.21.15.103:0/1922168153 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f27e40542a0 con 0x7f27fc103b80 2024-09-11T00:13:37.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.970+0000 7f27fb400640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27e003f510 0x7f27e00419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:37.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.970+0000 7f27fb400640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27e003f510 0x7f27e00419d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009fc0 tx=0x7f27ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:37.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:37.974+0000 7f27f3e00640 1 -- 172.21.15.103:0/1922168153 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f27e400b2c0 con 0x7f27fc103b80 2024-09-11T00:13:38.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.162+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f27fc063df0 con 0x7f27fc103b80 2024-09-11T00:13:38.168 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.167+0000 7f27f3e00640 1 -- 172.21.15.103:0/1922168153 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v23)=0 v23) ==== 108+0+0 (secure 0 0 0) 0x7f27e4014230 con 0x7f27fc103b80 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.169+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27e003f510 msgr2=0x7f27e00419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27e003f510 0x7f27e00419d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009fc0 tx=0x7f27ec005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc103b80 msgr2=0x7f27fc075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc103b80 0x7f27fc075540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f27e400e9f0 tx=0x7f27e400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 shutdown_connections 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27e003f510 0x7f27e00419d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27fc104540 0x7f27fc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27fc103b80 0x7f27fc075540 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 --2- 172.21.15.103:0/1922168153 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27fc102a20 0x7f27fc076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 >> 172.21.15.103:0/1922168153 conn(0x7f27fc0fe2f0 msgr2=0x7f27fc106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 shutdown_connections 2024-09-11T00:13:38.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:38.170+0000 7f2802600640 1 -- 172.21.15.103:0/1922168153 wait complete. 2024-09-11T00:13:39.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.462+0000 7fcdab000640 1 -- 172.21.15.103:0/1536491842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41043b0 msgr2=0x7fcda410ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.462+0000 7fcdab000640 1 --2- 172.21.15.103:0/1536491842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41043b0 0x7fcda410ad30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009cd0 tx=0x7fcd9c02a8e0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.463+0000 7fcdab000640 1 -- 172.21.15.103:0/1536491842 shutdown_connections 2024-09-11T00:13:39.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.463+0000 7fcdab000640 1 --2- 172.21.15.103:0/1536491842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41043b0 0x7fcda410ad30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.463+0000 7fcdab000640 1 --2- 172.21.15.103:0/1536491842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda41039f0 0x7fcda4103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.463+0000 7fcdab000640 1 --2- 172.21.15.103:0/1536491842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcda4102890 0x7fcda4102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.463+0000 7fcdab000640 1 -- 172.21.15.103:0/1536491842 >> 172.21.15.103:0/1536491842 conn(0x7fcda40fe180 msgr2=0x7fcda41005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.463+0000 7fcdab000640 1 -- 172.21.15.103:0/1536491842 shutdown_connections 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.463+0000 7fcdab000640 1 -- 172.21.15.103:0/1536491842 wait complete. 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.464+0000 7fcdab000640 1 Processor -- start 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.464+0000 7fcdab000640 1 -- start start 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.464+0000 7fcdab000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda4102890 0x7fcda4076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.464+0000 7fcdab000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41039f0 0x7fcda4075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcdab000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcda41043b0 0x7fcda4075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcdab000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcda410e420 con 0x7fcda41039f0 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcdab000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcda410e2a0 con 0x7fcda4102890 2024-09-11T00:13:39.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcdab000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcda410e5a0 con 0x7fcda41043b0 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41039f0 0x7fcda4075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41039f0 0x7fcda4075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33530/0 (socket says 172.21.15.103:33530) 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 -- 172.21.15.103:0/1650119712 learned_addr learned my addr 172.21.15.103:0/1650119712 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9c00640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda4102890 0x7fcda4076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcdaa600640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcda41043b0 0x7fcda4075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 -- 172.21.15.103:0/1650119712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcda41043b0 msgr2=0x7fcda4075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcda41043b0 0x7fcda4075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 -- 172.21.15.103:0/1650119712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda4102890 msgr2=0x7fcda4076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda4102890 0x7fcda4076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9200640 1 -- 172.21.15.103:0/1650119712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda4076340 con 0x7fcda41039f0 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.465+0000 7fcda9c00640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda4102890 0x7fcda4076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:39.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.466+0000 7fcda9200640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41039f0 0x7fcda4075540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000ea60 tx=0x7fcd9000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.466+0000 7fcda2000640 1 -- 172.21.15.103:0/1650119712 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd9000cde0 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.466+0000 7fcda2000640 1 -- 172.21.15.103:0/1650119712 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcd900035f0 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.466+0000 7fcda2000640 1 -- 172.21.15.103:0/1650119712 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90010690 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.466+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda41a2b40 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.466+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda41a3050 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.467+0000 7fcda2000640 1 -- 172.21.15.103:0/1650119712 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7fcd90004000 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.467+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda4109b40 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.468+0000 7fcda2000640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcd8c03f5c0 0x7fcd8c041a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.468+0000 7fcda2000640 1 -- 172.21.15.103:0/1650119712 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcd90054090 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.468+0000 7fcda9c00640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcd8c03f5c0 0x7fcd8c041a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.469+0000 7fcda9c00640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcd8c03f5c0 0x7fcd8c041a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009fc0 tx=0x7fcd98005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.473+0000 7fcda2000640 1 -- 172.21.15.103:0/1650119712 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcd9000f3c0 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.686+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fcda4063d80 con 0x7fcda41039f0 2024-09-11T00:13:39.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.691+0000 7fcda2000640 1 -- 172.21.15.103:0/1650119712 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fcd90021430 con 0x7fcda41039f0 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcd8c03f5c0 msgr2=0x7fcd8c041a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcd8c03f5c0 0x7fcd8c041a80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd98009fc0 tx=0x7fcd98005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41039f0 msgr2=0x7fcda4075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41039f0 0x7fcda4075540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000ea60 tx=0x7fcd9000ef20 comp rx=0 tx=0).stop 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 shutdown_connections 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcd8c03f5c0 0x7fcd8c041a80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcda41043b0 0x7fcda4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcda41039f0 0x7fcda4075540 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 --2- 172.21.15.103:0/1650119712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcda4102890 0x7fcda4076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 >> 172.21.15.103:0/1650119712 conn(0x7fcda40fe180 msgr2=0x7fcda4106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 shutdown_connections 2024-09-11T00:13:39.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:35.693+0000 7fcdab000640 1 -- 172.21.15.103:0/1650119712 wait complete. 2024-09-11T00:13:39.175 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 11 00:13:38 smithi117 systemd[1]: Started Ceph mgr.y for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:13:39.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:39 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:39.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:39 smithi103 ceph-mon[30578]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:39.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:39 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:39 smithi117 ceph-mon[35525]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:39 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:39 smithi183 ceph-mon[35934]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:39.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.858+0000 7f144a400640 1 -- 172.21.15.103:0/3643182436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 msgr2=0x7f1444104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.858+0000 7f144a400640 1 --2- 172.21.15.103:0/3643182436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444104010 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1430009930 tx=0x7f143002a420 comp rx=0 tx=0).stop 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.858+0000 7f144a400640 1 -- 172.21.15.103:0/3643182436 shutdown_connections 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.858+0000 7f144a400640 1 --2- 172.21.15.103:0/3643182436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444104550 0x7f144410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.858+0000 7f144a400640 1 --2- 172.21.15.103:0/3643182436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444104010 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.858+0000 7f144a400640 1 --2- 172.21.15.103:0/3643182436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1444102a30 0x7f1444102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.858+0000 7f144a400640 1 -- 172.21.15.103:0/3643182436 >> 172.21.15.103:0/3643182436 conn(0x7f14440fe320 msgr2=0x7f1444100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.859+0000 7f144a400640 1 -- 172.21.15.103:0/3643182436 shutdown_connections 2024-09-11T00:13:39.860 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.859+0000 7f144a400640 1 -- 172.21.15.103:0/3643182436 wait complete. 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 Processor -- start 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 -- start start 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102a30 0x7f1444076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1444104550 0x7f1444075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f144410e5a0 con 0x7f1444102a30 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f144410e420 con 0x7f1444103b90 2024-09-11T00:13:39.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.860+0000 7f144a400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f144410e720 con 0x7f1444104550 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1443400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102a30 0x7f1444076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:39930/0 (socket says 172.21.15.103:39930) 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 -- 172.21.15.103:0/2270765269 learned_addr learned my addr 172.21.15.103:0/2270765269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1443e00640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1444104550 0x7f1444075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 -- 172.21.15.103:0/2270765269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1444104550 msgr2=0x7f1444075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1444104550 0x7f1444075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 -- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102a30 msgr2=0x7f1444076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102a30 0x7f1444076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 -- 172.21.15.103:0/2270765269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1444076340 con 0x7f1444103b90 2024-09-11T00:13:39.863 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.861+0000 7f1442a00640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444075540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f143002a940 tx=0x7f1430003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:39.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.862+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143003f070 con 0x7f1444103b90 2024-09-11T00:13:39.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.862+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1430031b00 con 0x7f1444103b90 2024-09-11T00:13:39.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.862+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14441a2e10 con 0x7f1444103b90 2024-09-11T00:13:39.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.862+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430035030 con 0x7f1444103b90 2024-09-11T00:13:39.864 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.862+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441a3270 con 0x7f1444103b90 2024-09-11T00:13:39.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.863+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14441a34a0 con 0x7f1444103b90 2024-09-11T00:13:39.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.863+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) ==== 51481+0+0 (secure 0 0 0) 0x7f1430031670 con 0x7f1444103b90 2024-09-11T00:13:39.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.863+0000 7f143be00640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f142803f510 0x7f14280419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:39.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.863+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1430074790 con 0x7f1444103b90 2024-09-11T00:13:39.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.864+0000 7f1443400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f142803f510 0x7f14280419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:39.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.864+0000 7f1443400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f142803f510 0x7f14280419d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1434009bd0 tx=0x7f1434005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:39.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:39.873+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1430031cb0 con 0x7f1444103b90 2024-09-11T00:13:39.948 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:40.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.080+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f14441a3670 con 0x7f1444103b90 2024-09-11T00:13:40.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.089+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1430039ab0 con 0x7f1444103b90 2024-09-11T00:13:40.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.089+0000 7f143be00640 1 -- 172.21.15.103:0/2270765269 <== mon.2 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f1430039350 con 0x7f1444103b90 2024-09-11T00:13:40.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f142803f510 msgr2=0x7f14280419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:40.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f142803f510 0x7f14280419d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1434009bd0 tx=0x7f1434005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 msgr2=0x7f1444075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444075540 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f143002a940 tx=0x7f1430003b20 comp rx=0 tx=0).stop 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 shutdown_connections 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f142803f510 0x7f14280419d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1444104550 0x7f1444075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1444103b90 0x7f1444075540 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 --2- 172.21.15.103:0/2270765269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102a30 0x7f1444076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 >> 172.21.15.103:0/2270765269 conn(0x7f14440fe320 msgr2=0x7f1444104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 shutdown_connections 2024-09-11T00:13:40.093 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:40.091+0000 7f144a400640 1 -- 172.21.15.103:0/2270765269 wait complete. 2024-09-11T00:13:41.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:41.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:41.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:13:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:42.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:42 smithi103 ceph-mon[30578]: Standby manager daemon z started 2024-09-11T00:13:42.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:42 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-11T00:13:42.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:42 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:13:42.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:42 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-11T00:13:42.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:42 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:13:42.319 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:42.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:42 smithi117 ceph-mon[35525]: Standby manager daemon z started 2024-09-11T00:13:42.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:42 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-11T00:13:42.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:42 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:13:42.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:42 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-11T00:13:42.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:42 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:13:42.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:42.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:42 smithi183 ceph-mon[35934]: Standby manager daemon z started 2024-09-11T00:13:42.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:42 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-11T00:13:42.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:42 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:13:42.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:42 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-11T00:13:42.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:42 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.183:0/2379980332' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:13:42.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:13:42.672 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:43.401 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:43 smithi103 ceph-mon[30578]: mgrmap e14: x(active, since 75s), standbys: z 2024-09-11T00:13:43.441 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-11T00:13:43.442 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:43 smithi103 ceph-mon[30578]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:43.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:43 smithi117 ceph-mon[35525]: mgrmap e14: x(active, since 75s), standbys: z 2024-09-11T00:13:43.442 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-11T00:13:43.443 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:43 smithi117 ceph-mon[35525]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:43.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:43 smithi183 ceph-mon[35934]: mgrmap e14: x(active, since 75s), standbys: z 2024-09-11T00:13:43.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-11T00:13:43.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:43 smithi183 ceph-mon[35934]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:44.340 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:44 smithi103 ceph-mon[30578]: Standby manager daemon y started 2024-09-11T00:13:44.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:44 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-11T00:13:44.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:44 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:13:44.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:44 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-11T00:13:44.341 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:44 smithi103 ceph-mon[30578]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:13:44.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:44 smithi117 ceph-mon[35525]: Standby manager daemon y started 2024-09-11T00:13:44.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:44 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-11T00:13:44.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:44 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:13:44.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:44 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-11T00:13:44.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:44 smithi117 ceph-mon[35525]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:13:44.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:44 smithi183 ceph-mon[35934]: Standby manager daemon y started 2024-09-11T00:13:44.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:44 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-11T00:13:44.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:44 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-11T00:13:44.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:44 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-11T00:13:44.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:44 smithi183 ceph-mon[35934]: from='mgr.? 172.21.15.117:0/491140873' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-11T00:13:44.624 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi103') 2024-09-11T00:13:45.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 -- 172.21.15.103:0/3056955432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc208109250 msgr2=0x7fc208109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 --2- 172.21.15.103:0/3056955432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc208109250 0x7fc208109650 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009900 tx=0x7fc1f802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 -- 172.21.15.103:0/3056955432 shutdown_connections 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 --2- 172.21.15.103:0/3056955432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc20810ab40 0x7fc2081114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 --2- 172.21.15.103:0/3056955432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc20810a180 0x7fc20810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 --2- 172.21.15.103:0/3056955432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc208109250 0x7fc208109650 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 -- 172.21.15.103:0/3056955432 >> 172.21.15.103:0/3056955432 conn(0x7fc2080783a0 msgr2=0x7fc20807a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 -- 172.21.15.103:0/3056955432 shutdown_connections 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.241+0000 7fc20e400640 1 -- 172.21.15.103:0/3056955432 wait complete. 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.242+0000 7fc20e400640 1 Processor -- start 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.242+0000 7fc20e400640 1 -- start start 2024-09-11T00:13:45.256 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc20e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc208109250 0x7fc20819f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc20e400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc20810a180 0x7fc20819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc20e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc20810ab40 0x7fc2081a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc20e400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc208114d10 con 0x7fc20810a180 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc20e400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc208114b90 con 0x7fc208109250 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc20e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc208114e90 con 0x7fc20810ab40 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc20810a180 0x7fc20819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc20810a180 0x7fc20819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33746/0 (socket says 172.21.15.103:33746) 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 -- 172.21.15.103:0/2335118767 learned_addr learned my addr 172.21.15.103:0/2335118767 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc207400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc208109250 0x7fc20819f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc207e00640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc20810ab40 0x7fc2081a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 -- 172.21.15.103:0/2335118767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc20810ab40 msgr2=0x7fc2081a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc20810ab40 0x7fc2081a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 -- 172.21.15.103:0/2335118767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc208109250 msgr2=0x7fc20819f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc208109250 0x7fc20819f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.243+0000 7fc206a00640 1 -- 172.21.15.103:0/2335118767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2081a0ea0 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.244+0000 7fc206a00640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc20810a180 0x7fc20819f9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400d950 tx=0x7fc1f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.244+0000 7fc1ffe00640 1 -- 172.21.15.103:0/2335118767 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4017070 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.244+0000 7fc1ffe00640 1 -- 172.21.15.103:0/2335118767 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f4009c30 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.244+0000 7fc1ffe00640 1 -- 172.21.15.103:0/2335118767 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f400f620 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.244+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2081a1190 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.244+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2081a7870 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.246+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc208109650 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.248+0000 7fc1ffe00640 1 -- 172.21.15.103:0/2335118767 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7fc1f4010460 con 0x7fc20810a180 2024-09-11T00:13:45.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.250+0000 7fc1ffe00640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc1ec0b52d0 0x7fc1ec0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.250+0000 7fc1ffe00640 1 -- 172.21.15.103:0/2335118767 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc1f40e3b80 con 0x7fc20810a180 2024-09-11T00:13:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.250+0000 7fc207400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc1ec0b52d0 0x7fc1ec0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.250+0000 7fc207400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc1ec0b52d0 0x7fc1ec0b7790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005ab0 tx=0x7fc1f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:45.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.253+0000 7fc1ffe00640 1 -- 172.21.15.103:0/2335118767 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc1f40ac670 con 0x7fc20810a180 2024-09-11T00:13:45.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 -- 172.21.15.103:0/3844449298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8105dc0 msgr2=0x7f13c8106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 --2- 172.21.15.103:0/3844449298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8105dc0 0x7f13c8106240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009930 tx=0x7f13b002a440 comp rx=0 tx=0).stop 2024-09-11T00:13:45.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 -- 172.21.15.103:0/3844449298 shutdown_connections 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 --2- 172.21.15.103:0/3844449298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13c8106780 0x7f13c810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 --2- 172.21.15.103:0/3844449298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8105dc0 0x7f13c8106240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 --2- 172.21.15.103:0/3844449298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13c8104c60 0x7f13c8105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 -- 172.21.15.103:0/3844449298 >> 172.21.15.103:0/3844449298 conn(0x7f13c8100510 msgr2=0x7f13c8102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 -- 172.21.15.103:0/3844449298 shutdown_connections 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.338+0000 7f13cda00640 1 -- 172.21.15.103:0/3844449298 wait complete. 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.339+0000 7f13cda00640 1 Processor -- start 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.339+0000 7f13cda00640 1 -- start start 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13cda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8104c60 0x7f13c819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13cda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13c8105dc0 0x7f13c819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13cda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13c8106780 0x7f13c819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13cda00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8110930 con 0x7f13c8104c60 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13cda00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f13c81107b0 con 0x7f13c8105dc0 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13cda00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8110ab0 con 0x7f13c8106780 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13c6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8104c60 0x7f13c819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13c6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8104c60 0x7f13c819b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33718/0 (socket says 172.21.15.103:33718) 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13c8105dc0 0x7f13c819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13c6a00640 1 -- 172.21.15.103:0/3586318901 learned_addr learned my addr 172.21.15.103:0/3586318901 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.340+0000 7f13c7400640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13c8106780 0x7f13c819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13c6a00640 1 -- 172.21.15.103:0/3586318901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13c8106780 msgr2=0x7f13c819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13c6a00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13c8106780 0x7f13c819c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13c6a00640 1 -- 172.21.15.103:0/3586318901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13c8105dc0 msgr2=0x7f13c819b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13c6a00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13c8105dc0 0x7f13c819b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13c6a00640 1 -- 172.21.15.103:0/3586318901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c819cbe0 con 0x7f13c8104c60 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13bfe00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13c8105dc0 0x7f13c819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13c6a00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8104c60 0x7f13c819b0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d770 tx=0x7f13b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13bf400640 1 -- 172.21.15.103:0/3586318901 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8017070 con 0x7f13c8104c60 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13bf400640 1 -- 172.21.15.103:0/3586318901 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b80054d0 con 0x7f13c8104c60 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c819ced0 con 0x7f13c8104c60 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13bf400640 1 -- 172.21.15.103:0/3586318901 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8004c30 con 0x7f13c8104c60 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.341+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a35b0 con 0x7f13c8104c60 2024-09-11T00:13:45.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.342+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c810bf60 con 0x7f13c8104c60 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.345+0000 7f13bf400640 1 -- 172.21.15.103:0/3586318901 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f13b8004dd0 con 0x7f13c8104c60 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.346+0000 7f13bf400640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f13ac0b5250 0x7f13ac0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.346+0000 7f13bf400640 1 -- 172.21.15.103:0/3586318901 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f13b80e3e40 con 0x7f13c8104c60 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.346+0000 7f13bfe00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f13ac0b5250 0x7f13ac0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.347+0000 7f13bfe00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f13ac0b5250 0x7f13ac0b7710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005ab0 tx=0x7f13b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.348+0000 7f13bf400640 1 -- 172.21.15.103:0/3586318901 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f13b80ac930 con 0x7f13c8104c60 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.537+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f13c802e870 con 0x7f13c8104c60 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.543+0000 7f13bf400640 1 -- 172.21.15.103:0/3586318901 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f13b80ac230 con 0x7f13c8104c60 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f13ac0b5250 msgr2=0x7f13ac0b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f13ac0b5250 0x7f13ac0b7710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005ab0 tx=0x7f13b0005a20 comp rx=0 tx=0).stop 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8104c60 msgr2=0x7f13c819b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8104c60 0x7f13c819b0b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13b800d770 tx=0x7f13b800dc30 comp rx=0 tx=0).stop 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 shutdown_connections 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f13ac0b5250 0x7f13ac0b7710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13c8106780 0x7f13c819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13c8105dc0 0x7f13c819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 --2- 172.21.15.103:0/3586318901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13c8104c60 0x7f13c819b0b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 >> 172.21.15.103:0/3586318901 conn(0x7f13c8100510 msgr2=0x7f13c8109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 shutdown_connections 2024-09-11T00:13:45.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:44.545+0000 7f13cda00640 1 -- 172.21.15.103:0/3586318901 wait complete. 2024-09-11T00:13:45.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:45 smithi103 ceph-mon[30578]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-09-11T00:13:45.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-11T00:13:45.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:45 smithi103 ceph-mon[30578]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:45.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:45 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3586318901' entity='client.admin' 2024-09-11T00:13:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:45 smithi117 ceph-mon[35525]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-09-11T00:13:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-11T00:13:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:45 smithi117 ceph-mon[35525]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:45 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3586318901' entity='client.admin' 2024-09-11T00:13:45.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.443+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fc2081102d0 con 0x7fc20810a180 2024-09-11T00:13:45.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.447+0000 7fc1ffe00640 1 -- 172.21.15.103:0/2335118767 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v26)=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fc1f4009de0 con 0x7fc20810a180 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc1ec0b52d0 msgr2=0x7fc1ec0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc1ec0b52d0 0x7fc1ec0b7790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005ab0 tx=0x7fc1f8005a20 comp rx=0 tx=0).stop 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc20810a180 msgr2=0x7fc20819f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc20810a180 0x7fc20819f9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc1f400d950 tx=0x7fc1f400de10 comp rx=0 tx=0).stop 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 shutdown_connections 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc1ec0b52d0 0x7fc1ec0b7790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc20810ab40 0x7fc2081a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc20810a180 0x7fc20819f9c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 --2- 172.21.15.103:0/2335118767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc208109250 0x7fc20819f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:45.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 >> 172.21.15.103:0/2335118767 conn(0x7fc2080783a0 msgr2=0x7fc20810d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 shutdown_connections 2024-09-11T00:13:45.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:45.450+0000 7fc20e400640 1 -- 172.21.15.103:0/2335118767 wait complete. 2024-09-11T00:13:45.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:45 smithi183 ceph-mon[35934]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-09-11T00:13:45.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-11T00:13:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:45 smithi183 ceph-mon[35934]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:45 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3586318901' entity='client.admin' 2024-09-11T00:13:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:46 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2335118767' entity='client.admin' 2024-09-11T00:13:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:46 smithi183 ceph-mon[35934]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:46.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:46 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2335118767' entity='client.admin' 2024-09-11T00:13:46.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:46 smithi103 ceph-mon[30578]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:46 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2335118767' entity='client.admin' 2024-09-11T00:13:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:46 smithi117 ceph-mon[35525]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:47.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.578+0000 7fc709800640 1 -- 172.21.15.103:0/3395849819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 msgr2=0x7fc704104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:47.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.578+0000 7fc709800640 1 --2- 172.21.15.103:0/3395849819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc704104010 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009930 tx=0x7fc6ec02a420 comp rx=0 tx=0).stop 2024-09-11T00:13:47.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.578+0000 7fc709800640 1 -- 172.21.15.103:0/3395849819 shutdown_connections 2024-09-11T00:13:47.702 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.578+0000 7fc709800640 1 --2- 172.21.15.103:0/3395849819 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc704104550 0x7fc70410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.578+0000 7fc709800640 1 --2- 172.21.15.103:0/3395849819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc704104010 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.578+0000 7fc709800640 1 --2- 172.21.15.103:0/3395849819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc704102a30 0x7fc704102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.579+0000 7fc709800640 1 -- 172.21.15.103:0/3395849819 >> 172.21.15.103:0/3395849819 conn(0x7fc7040fe320 msgr2=0x7fc704100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.579+0000 7fc709800640 1 -- 172.21.15.103:0/3395849819 shutdown_connections 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.579+0000 7fc709800640 1 -- 172.21.15.103:0/3395849819 wait complete. 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.579+0000 7fc709800640 1 Processor -- start 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc709800640 1 -- start start 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc709800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc704102a30 0x7fc70419b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc709800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc70419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc709800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc704104550 0x7fc70419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc709800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410e700 con 0x7fc704103b90 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc709800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410e580 con 0x7fc704102a30 2024-09-11T00:13:47.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc709800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc70410e880 con 0x7fc704104550 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc6fa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc70419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc6fa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc70419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33764/0 (socket says 172.21.15.103:33764) 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc6fa000640 1 -- 172.21.15.103:0/1120460585 learned_addr learned my addr 172.21.15.103:0/1120460585 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc703400640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc704104550 0x7fc70419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.580+0000 7fc702a00640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc704102a30 0x7fc70419b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc703400640 1 -- 172.21.15.103:0/1120460585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc704102a30 msgr2=0x7fc70419b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc703400640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc704102a30 0x7fc70419b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc703400640 1 -- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 msgr2=0x7fc70419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc703400640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc70419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc703400640 1 -- 172.21.15.103:0/1120460585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70419cb60 con 0x7fc704104550 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc6fa000640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc70419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:47.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc703400640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc704104550 0x7fc70419c2c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00da10 tx=0x7fc6fc00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc6fbe00640 1 -- 172.21.15.103:0/1120460585 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc00bd70 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70419ce50 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc6fbe00640 1 -- 172.21.15.103:0/1120460585 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6fc003e20 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc6fbe00640 1 -- 172.21.15.103:0/1120460585 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc0106c0 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.581+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7041a3530 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.582+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7041a3700 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.585+0000 7fc6fbe00640 1 -- 172.21.15.103:0/1120460585 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7fc6fc024060 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.586+0000 7fc6fbe00640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc6e80b5250 0x7fc6e80b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.586+0000 7fc6fbe00640 1 -- 172.21.15.103:0/1120460585 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc6fc0e43c0 con 0x7fc704104550 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.587+0000 7fc702a00640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc6e80b5250 0x7fc6e80b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.587+0000 7fc702a00640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc6e80b5250 0x7fc6e80b7710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009bd0 tx=0x7fc6f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:47.705 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.588+0000 7fc6fbe00640 1 -- 172.21.15.103:0/1120460585 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc6fc0aceb0 con 0x7fc704104550 2024-09-11T00:13:47.709 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:47.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.781+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fc7041a3a80 con 0x7fc704104550 2024-09-11T00:13:47.788 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.787+0000 7fc6fbe00640 1 -- 172.21.15.103:0/1120460585 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v27)=0 v27) ==== 127+0+0 (secure 0 0 0) 0x7fc6fc0ac7b0 con 0x7fc704104550 2024-09-11T00:13:47.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc6e80b5250 msgr2=0x7fc6e80b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:47.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc6e80b5250 0x7fc6e80b7710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009bd0 tx=0x7fc6f4005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:47.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc704104550 msgr2=0x7fc70419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:47.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc704104550 0x7fc70419c2c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc00da10 tx=0x7fc6fc00ded0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 shutdown_connections 2024-09-11T00:13:47.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc6e80b5250 0x7fc6e80b7710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.791 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc704104550 0x7fc70419c2c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc704103b90 0x7fc70419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 --2- 172.21.15.103:0/1120460585 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc704102a30 0x7fc70419b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:47.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 >> 172.21.15.103:0/1120460585 conn(0x7fc7040fe320 msgr2=0x7fc704075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:47.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 shutdown_connections 2024-09-11T00:13:47.792 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:47.789+0000 7fc709800640 1 -- 172.21.15.103:0/1120460585 wait complete. 2024-09-11T00:13:48.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.290+0000 7fcf98e00640 1 -- 172.21.15.103:0/4220150733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106760 msgr2=0x7fcf9410d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.290+0000 7fcf98e00640 1 --2- 172.21.15.103:0/4220150733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106760 0x7fcf9410d0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009cd0 tx=0x7fcf8c02a8e0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.291+0000 7fcf98e00640 1 -- 172.21.15.103:0/4220150733 shutdown_connections 2024-09-11T00:13:48.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.291+0000 7fcf98e00640 1 --2- 172.21.15.103:0/4220150733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106760 0x7fcf9410d0e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.291+0000 7fcf98e00640 1 --2- 172.21.15.103:0/4220150733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94105da0 0x7fcf94106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.291+0000 7fcf98e00640 1 --2- 172.21.15.103:0/4220150733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf94104c40 0x7fcf94105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.291+0000 7fcf98e00640 1 -- 172.21.15.103:0/4220150733 >> 172.21.15.103:0/4220150733 conn(0x7fcf94100510 msgr2=0x7fcf94102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.291+0000 7fcf98e00640 1 -- 172.21.15.103:0/4220150733 shutdown_connections 2024-09-11T00:13:48.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.291+0000 7fcf98e00640 1 -- 172.21.15.103:0/4220150733 wait complete. 2024-09-11T00:13:48.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 Processor -- start 2024-09-11T00:13:48.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 -- start start 2024-09-11T00:13:48.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104c40 0x7fcf9419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94105da0 0x7fcf9419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf94106760 0x7fcf9419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94110910 con 0x7fcf94105da0 2024-09-11T00:13:48.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94110790 con 0x7fcf94104c40 2024-09-11T00:13:48.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf98e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94110a90 con 0x7fcf94106760 2024-09-11T00:13:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf91400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94105da0 0x7fcf9419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf91400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94105da0 0x7fcf9419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33784/0 (socket says 172.21.15.103:33784) 2024-09-11T00:13:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf91400640 1 -- 172.21.15.103:0/639197130 learned_addr learned my addr 172.21.15.103:0/639197130 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf91e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104c40 0x7fcf9419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.292+0000 7fcf92800640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf94106760 0x7fcf9419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf91400640 1 -- 172.21.15.103:0/639197130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf94106760 msgr2=0x7fcf9419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf91400640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf94106760 0x7fcf9419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf91400640 1 -- 172.21.15.103:0/639197130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104c40 msgr2=0x7fcf9419b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf91400640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104c40 0x7fcf9419b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf91400640 1 -- 172.21.15.103:0/639197130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9419cb70 con 0x7fcf94105da0 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf92800640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf94106760 0x7fcf9419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf91e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104c40 0x7fcf9419b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf91400640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94105da0 0x7fcf9419b690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000e9f0 tx=0x7fcf8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf8aa00640 1 -- 172.21.15.103:0/639197130 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8000cd80 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf8aa00640 1 -- 172.21.15.103:0/639197130 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf800108e0 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf8aa00640 1 -- 172.21.15.103:0/639197130 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80018980 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9419ce60 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.293+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941a3540 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.294+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf94105040 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.297+0000 7fcf8aa00640 1 -- 172.21.15.103:0/639197130 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7fcf80010430 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.298+0000 7fcf8aa00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52d0 0x7fcf780b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.299+0000 7fcf8aa00640 1 -- 172.21.15.103:0/639197130 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcf800e4270 con 0x7fcf94105da0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.299+0000 7fcf91e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52d0 0x7fcf780b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.299+0000 7fcf91e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52d0 0x7fcf780b7790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009fc0 tx=0x7fcf84005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:48.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.303+0000 7fcf8aa00640 1 -- 172.21.15.103:0/639197130 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcf800acd60 con 0x7fcf94105da0 2024-09-11T00:13:48.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.496+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fcf94063d50 con 0x7fcf94105da0 2024-09-11T00:13:48.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.501+0000 7fcf8aa00640 1 -- 172.21.15.103:0/639197130 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v28)=0 v28) ==== 120+0+0 (secure 0 0 0) 0x7fcf800ac660 con 0x7fcf94105da0 2024-09-11T00:13:48.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52d0 msgr2=0x7fcf780b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52d0 0x7fcf780b7790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009fc0 tx=0x7fcf84005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:48.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94105da0 msgr2=0x7fcf9419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:48.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94105da0 0x7fcf9419b690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000e9f0 tx=0x7fcf8000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 shutdown_connections 2024-09-11T00:13:48.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52d0 0x7fcf780b7790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf94106760 0x7fcf9419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94105da0 0x7fcf9419b690 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 --2- 172.21.15.103:0/639197130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94104c40 0x7fcf9419b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:48.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 >> 172.21.15.103:0/639197130 conn(0x7fcf94100510 msgr2=0x7fcf94109110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:48.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.503+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 shutdown_connections 2024-09-11T00:13:48.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.504+0000 7fcf98e00640 1 -- 172.21.15.103:0/639197130 wait complete. 2024-09-11T00:13:48.978 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:48 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:48.978 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:48 smithi103 ceph-mon[30578]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:48.978 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:48 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/639197130' entity='client.admin' 2024-09-11T00:13:49.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:48 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:49.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:48 smithi117 ceph-mon[35525]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:49.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:48 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/639197130' entity='client.admin' 2024-09-11T00:13:49.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:48 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:49.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:48 smithi183 ceph-mon[35934]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:49.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:48 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/639197130' entity='client.admin' 2024-09-11T00:13:49.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 -- 172.21.15.103:0/597270843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b14102880 msgr2=0x7f6b14102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:49.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 --2- 172.21.15.103:0/597270843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b14102880 0x7f6b14102c80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009930 tx=0x7f6b0402a440 comp rx=0 tx=0).stop 2024-09-11T00:13:49.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 -- 172.21.15.103:0/597270843 shutdown_connections 2024-09-11T00:13:49.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 --2- 172.21.15.103:0/597270843 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b141043a0 0x7f6b1410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 --2- 172.21.15.103:0/597270843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141039e0 0x7f6b14103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 --2- 172.21.15.103:0/597270843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b14102880 0x7f6b14102c80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 -- 172.21.15.103:0/597270843 >> 172.21.15.103:0/597270843 conn(0x7f6b140fe150 msgr2=0x7f6b14100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:49.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 -- 172.21.15.103:0/597270843 shutdown_connections 2024-09-11T00:13:49.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.490+0000 7f6b13e00640 1 -- 172.21.15.103:0/597270843 wait complete. 2024-09-11T00:13:49.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.491+0000 7f6b13e00640 1 Processor -- start 2024-09-11T00:13:49.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.491+0000 7f6b13e00640 1 -- start start 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b13e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b14102880 0x7f6b1419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b13e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b141039e0 0x7f6b1419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b13e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141043a0 0x7f6b1419c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b13e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410e550 con 0x7f6b141039e0 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b13e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410e3d0 con 0x7f6b14102880 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b13e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410e6d0 con 0x7f6b141043a0 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b0be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b141039e0 0x7f6b1419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b0be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b141039e0 0x7f6b1419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33828/0 (socket says 172.21.15.103:33828) 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b0be00640 1 -- 172.21.15.103:0/4165785253 learned_addr learned my addr 172.21.15.103:0/4165785253 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b12a00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b14102880 0x7f6b1419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b13400640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141043a0 0x7f6b1419c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:49.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b12a00640 1 -- 172.21.15.103:0/4165785253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141043a0 msgr2=0x7f6b1419c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:49.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b12a00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141043a0 0x7f6b1419c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.492+0000 7f6b12a00640 1 -- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b141039e0 msgr2=0x7f6b1419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:49.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b12a00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b141039e0 0x7f6b1419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b12a00640 1 -- 172.21.15.103:0/4165785253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1419c8b0 con 0x7f6b14102880 2024-09-11T00:13:49.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b0be00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b141039e0 0x7f6b1419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:13:49.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b12a00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b14102880 0x7f6b1419ad80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b0402a960 tx=0x7f6b04003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:49.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b0b400640 1 -- 172.21.15.103:0/4165785253 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0403f070 con 0x7f6b14102880 2024-09-11T00:13:49.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b0b400640 1 -- 172.21.15.103:0/4165785253 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b04031be0 con 0x7f6b14102880 2024-09-11T00:13:49.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1419cb40 con 0x7f6b14102880 2024-09-11T00:13:49.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b0b400640 1 -- 172.21.15.103:0/4165785253 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04035030 con 0x7f6b14102880 2024-09-11T00:13:49.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.493+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141a3280 con 0x7f6b14102880 2024-09-11T00:13:49.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.494+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14102c80 con 0x7f6b14102880 2024-09-11T00:13:49.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.496+0000 7f6b0b400640 1 -- 172.21.15.103:0/4165785253 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f6b04031730 con 0x7f6b14102880 2024-09-11T00:13:49.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.498+0000 7f6b0b400640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6af80b5370 0x7f6af80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:49.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.498+0000 7f6b0b400640 1 -- 172.21.15.103:0/4165785253 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6b041048b0 con 0x7f6b14102880 2024-09-11T00:13:49.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.498+0000 7f6b0be00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6af80b5370 0x7f6af80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:49.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.498+0000 7f6b0be00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6af80b5370 0x7f6af80b7830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6afc011500 tx=0x7f6afc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:49.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.501+0000 7f6b0b400640 1 -- 172.21.15.103:0/4165785253 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6b040ccd50 con 0x7f6b14102880 2024-09-11T00:13:49.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.690+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f6b141a3450 con 0x7f6b14102880 2024-09-11T00:13:49.697 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.695+0000 7f6b0b400640 1 -- 172.21.15.103:0/4165785253 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v30)=0 v30) ==== 116+0+0 (secure 0 0 0) 0x7f6b040d1780 con 0x7f6b14102880 2024-09-11T00:13:49.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6af80b5370 msgr2=0x7f6af80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:49.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6af80b5370 0x7f6af80b7830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6afc011500 tx=0x7f6afc005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:49.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b14102880 msgr2=0x7f6b1419ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:49.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b14102880 0x7f6b1419ad80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6b0402a960 tx=0x7f6b04003b60 comp rx=0 tx=0).stop 2024-09-11T00:13:49.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 shutdown_connections 2024-09-11T00:13:49.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6af80b5370 0x7f6af80b7830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6b141043a0 0x7f6b1419c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b141039e0 0x7f6b1419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 --2- 172.21.15.103:0/4165785253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b14102880 0x7f6b1419ad80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:49.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 >> 172.21.15.103:0/4165785253 conn(0x7f6b140fe150 msgr2=0x7f6b14075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:49.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 shutdown_connections 2024-09-11T00:13:49.701 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:49.698+0000 7f6b13e00640 1 -- 172.21.15.103:0/4165785253 wait complete. 2024-09-11T00:13:49.844 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi103') 2024-09-11T00:13:49.845 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi103') 2024-09-11T00:13:49.845 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi103') 2024-09-11T00:13:49.845 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi103') 2024-09-11T00:13:49.846 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi103') 2024-09-11T00:13:49.846 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi103') 2024-09-11T00:13:50.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 -- 172.21.15.103:0/2773915237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10105c00 msgr2=0x7f9d10106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:50.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/2773915237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10105c00 0x7f9d10106080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8009930 tx=0x7f9cf802a440 comp rx=0 tx=0).stop 2024-09-11T00:13:50.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 -- 172.21.15.103:0/2773915237 shutdown_connections 2024-09-11T00:13:50.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/2773915237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d101065c0 0x7f9d1010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/2773915237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10105c00 0x7f9d10106080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/2773915237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d10104aa0 0x7f9d10104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 -- 172.21.15.103:0/2773915237 >> 172.21.15.103:0/2773915237 conn(0x7f9d10100370 msgr2=0x7f9d101027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 -- 172.21.15.103:0/2773915237 shutdown_connections 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.726+0000 7f9d0fe00640 1 -- 172.21.15.103:0/2773915237 wait complete. 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 Processor -- start 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 -- start start 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10104aa0 0x7f9d1019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d10105c00 0x7f9d1019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d101065c0 0x7f9d1019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d10110750 con 0x7f9d10104aa0 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101105d0 con 0x7f9d10105c00 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.727+0000 7f9d0fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9d101108d0 con 0x7f9d101065c0 2024-09-11T00:13:50.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d10105c00 0x7f9d1019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d0ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10104aa0 0x7f9d1019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d10105c00 0x7f9d1019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:47902/0 (socket says 172.21.15.103:47902) 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 -- 172.21.15.103:0/1849982060 learned_addr learned my addr 172.21.15.103:0/1849982060 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d0f400640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d101065c0 0x7f9d1019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 -- 172.21.15.103:0/1849982060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d101065c0 msgr2=0x7f9d1019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d101065c0 0x7f9d1019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 -- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10104aa0 msgr2=0x7f9d1019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10104aa0 0x7f9d1019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 -- 172.21.15.103:0/1849982060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1019c8f0 con 0x7f9d10105c00 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d0f400640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d101065c0 0x7f9d1019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.728+0000 7f9d06000640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d10105c00 0x7f9d1019b410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8031490 tx=0x7f9cf8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.729+0000 7f9d07e00640 1 -- 172.21.15.103:0/1849982060 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf803f070 con 0x7f9d10105c00 2024-09-11T00:13:50.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.729+0000 7f9d07e00640 1 -- 172.21.15.103:0/1849982060 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cf8031bf0 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.729+0000 7f9d07e00640 1 -- 172.21.15.103:0/1849982060 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8035030 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.729+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1019cb80 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.729+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d101a30f0 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.730+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1010bd50 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.732+0000 7f9d07e00640 1 -- 172.21.15.103:0/1849982060 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f9cf803b460 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.734+0000 7f9d07e00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cf40b5370 0x7f9cf40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.734+0000 7f9d07e00640 1 -- 172.21.15.103:0/1849982060 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9cf8104ff0 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.734+0000 7f9d0ea00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cf40b5370 0x7f9cf40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.735+0000 7f9d0ea00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cf40b5370 0x7f9cf40b7830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009bd0 tx=0x7f9d00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.735+0000 7f9d07e00640 1 -- 172.21.15.103:0/1849982060 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9cf80cdae0 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.930+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f9d10063d50 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.936+0000 7f9d07e00640 1 -- 172.21.15.103:0/1849982060 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f9cf8031da0 con 0x7f9d10105c00 2024-09-11T00:13:50.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cf40b5370 msgr2=0x7f9cf40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cf40b5370 0x7f9cf40b7830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d00009bd0 tx=0x7f9d00005f90 comp rx=0 tx=0).stop 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d10105c00 msgr2=0x7f9d1019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d10105c00 0x7f9d1019b410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8031490 tx=0x7f9cf8003c10 comp rx=0 tx=0).stop 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 shutdown_connections 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cf40b5370 0x7f9cf40b7830 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9d101065c0 0x7f9d1019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d10105c00 0x7f9d1019b410 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 --2- 172.21.15.103:0/1849982060 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d10104aa0 0x7f9d1019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:50.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 >> 172.21.15.103:0/1849982060 conn(0x7f9d10100370 msgr2=0x7f9d10108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:50.129 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:49 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:50.129 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:49 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' 2024-09-11T00:13:50.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 shutdown_connections 2024-09-11T00:13:50.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:48.938+0000 7f9d0fe00640 1 -- 172.21.15.103:0/1849982060 wait complete. 2024-09-11T00:13:50.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:49 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:50.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:49 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' 2024-09-11T00:13:50.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:49 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:50.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:49 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' 2024-09-11T00:13:50.818 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:50.856 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi103') 2024-09-11T00:13:50.857 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi103') 2024-09-11T00:13:51.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:50 smithi103 ceph-mon[30578]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:51.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:50 smithi183 ceph-mon[35934]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:51.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:50 smithi117 ceph-mon[35525]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:51.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi103') 2024-09-11T00:13:51.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi103') 2024-09-11T00:13:51.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi103') 2024-09-11T00:13:51.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi103') 2024-09-11T00:13:51.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi103') 2024-09-11T00:13:51.662 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi103') 2024-09-11T00:13:52.150 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi103') 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 -- 172.21.15.103:0/1028700332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1039e0 msgr2=0x7f515c103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 --2- 172.21.15.103:0/1028700332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1039e0 0x7f515c103e60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5148009930 tx=0x7f514802a420 comp rx=0 tx=0).stop 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 -- 172.21.15.103:0/1028700332 shutdown_connections 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 --2- 172.21.15.103:0/1028700332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c1043a0 0x7f515c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 --2- 172.21.15.103:0/1028700332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1039e0 0x7f515c103e60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 --2- 172.21.15.103:0/1028700332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c102880 0x7f515c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 -- 172.21.15.103:0/1028700332 >> 172.21.15.103:0/1028700332 conn(0x7f515c0fe150 msgr2=0x7f515c100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 -- 172.21.15.103:0/1028700332 shutdown_connections 2024-09-11T00:13:52.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.654+0000 7f5161200640 1 -- 172.21.15.103:0/1028700332 wait complete. 2024-09-11T00:13:52.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.655+0000 7f5161200640 1 Processor -- start 2024-09-11T00:13:52.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.655+0000 7f5161200640 1 -- start start 2024-09-11T00:13:52.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f5161200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c102880 0x7f515c076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f5161200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c1039e0 0x7f515c075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f5161200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1043a0 0x7f515c075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f5161200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10e530 con 0x7f515c1043a0 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f5161200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10e3b0 con 0x7f515c102880 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f5161200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10e6b0 con 0x7f515c1039e0 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1043a0 0x7f515c075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c102880 0x7f515c076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c102880 0x7f515c076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52692/0 (socket says 172.21.15.103:52692) 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515b400640 1 -- 172.21.15.103:0/2879314107 learned_addr learned my addr 172.21.15.103:0/2879314107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515aa00640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c1039e0 0x7f515c075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515be00640 1 -- 172.21.15.103:0/2879314107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c1039e0 msgr2=0x7f515c075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515be00640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c1039e0 0x7f515c075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515be00640 1 -- 172.21.15.103:0/2879314107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c102880 msgr2=0x7f515c076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515be00640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c102880 0x7f515c076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.656+0000 7f515be00640 1 -- 172.21.15.103:0/2879314107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c076340 con 0x7f515c1043a0 2024-09-11T00:13:52.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.657+0000 7f515be00640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1043a0 0x7f515c075a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f515400e970 tx=0x7f515400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:52.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.657+0000 7f5153e00640 1 -- 172.21.15.103:0/2879314107 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515400ccc0 con 0x7f515c1043a0 2024-09-11T00:13:52.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.657+0000 7f5153e00640 1 -- 172.21.15.103:0/2879314107 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5154005020 con 0x7f515c1043a0 2024-09-11T00:13:52.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.657+0000 7f5153e00640 1 -- 172.21.15.103:0/2879314107 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154004a00 con 0x7f515c1043a0 2024-09-11T00:13:52.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.657+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c1a2c50 con 0x7f515c1043a0 2024-09-11T00:13:52.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.657+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a3160 con 0x7f515c1043a0 2024-09-11T00:13:52.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.658+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c109b30 con 0x7f515c1043a0 2024-09-11T00:13:52.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.660+0000 7f5153e00640 1 -- 172.21.15.103:0/2879314107 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f5154004ba0 con 0x7f515c1043a0 2024-09-11T00:13:52.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.661+0000 7f5153e00640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51400b5370 0x7f51400b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:13:52.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.662+0000 7f5153e00640 1 -- 172.21.15.103:0/2879314107 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f51540e3e60 con 0x7f515c1043a0 2024-09-11T00:13:52.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.662+0000 7f515b400640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51400b5370 0x7f51400b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:13:52.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.662+0000 7f515b400640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51400b5370 0x7f51400b7830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f514c009fd0 tx=0x7f514c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:13:52.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.665+0000 7f5153e00640 1 -- 172.21.15.103:0/2879314107 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f51540e7030 con 0x7f515c1043a0 2024-09-11T00:13:52.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.849+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f515c063d80 con 0x7f515c1043a0 2024-09-11T00:13:52.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.853+0000 7f5153e00640 1 -- 172.21.15.103:0/2879314107 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v31)=0 v31) ==== 120+0+0 (secure 0 0 0) 0x7f51540ac950 con 0x7f515c1043a0 2024-09-11T00:13:52.858 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51400b5370 msgr2=0x7f51400b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51400b5370 0x7f51400b7830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f514c009fd0 tx=0x7f514c009500 comp rx=0 tx=0).stop 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1043a0 msgr2=0x7f515c075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1043a0 0x7f515c075a80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f515400e970 tx=0x7f515400ee30 comp rx=0 tx=0).stop 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 shutdown_connections 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51400b5370 0x7f51400b7830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f515c1043a0 0x7f515c075a80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f515c1039e0 0x7f515c075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 --2- 172.21.15.103:0/2879314107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f515c102880 0x7f515c076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 >> 172.21.15.103:0/2879314107 conn(0x7f515c0fe150 msgr2=0x7f515c106d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.856+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 shutdown_connections 2024-09-11T00:13:52.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:13:52.857+0000 7f5161200640 1 -- 172.21.15.103:0/2879314107 wait complete. 2024-09-11T00:13:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:53 smithi117 ceph-mon[35525]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:53 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2879314107' entity='client.admin' 2024-09-11T00:13:53.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:53 smithi103 ceph-mon[30578]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:53.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:53 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2879314107' entity='client.admin' 2024-09-11T00:13:53.696 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi103') 2024-09-11T00:13:53.696 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi103') 2024-09-11T00:13:53.697 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi103') 2024-09-11T00:13:53.697 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '40f10072-6fd2-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi103') 2024-09-11T00:13:53.697 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T00:13:53.697 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:13:53.697 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T00:13:53.734 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T00:13:53.734 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-09-11T00:13:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:53 smithi183 ceph-mon[35934]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:53 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2879314107' entity='client.admin' 2024-09-11T00:13:53.810 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T00:13:53.810 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:53.810 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-11T00:13:53.810 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:53.810 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:53.810 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-11 00:12:43.949171549 +0000 2024-09-11T00:13:53.810 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-11 00:08:08.286286397 +0000 2024-09-11T00:13:53.811 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-11 00:08:08.286286397 +0000 2024-09-11T00:13:53.811 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-11 00:08:08.286286397 +0000 2024-09-11T00:13:53.811 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T00:13:53.905 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-11T00:13:53.905 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-11T00:13:53.905 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.0002815 s, 1.8 MB/s 2024-09-11T00:13:53.908 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T00:13:53.944 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-09-11T00:13:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T00:13:54.020 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:54.020 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-09-11T00:13:54.020 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:54.020 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:54.020 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-11 00:12:43.993170675 +0000 2024-09-11T00:13:54.020 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-11 00:08:08.776276541 +0000 2024-09-11T00:13:54.021 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-11 00:08:08.776276541 +0000 2024-09-11T00:13:54.021 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-11 00:08:08.776276541 +0000 2024-09-11T00:13:54.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T00:13:54.114 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-11T00:13:54.115 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-11T00:13:54.115 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000287177 s, 1.8 MB/s 2024-09-11T00:13:54.117 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T00:13:54.154 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-09-11T00:13:54.231 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T00:13:54.231 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:54.231 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-09-11T00:13:54.231 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:54.231 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:54.231 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-11 00:12:44.037169801 +0000 2024-09-11T00:13:54.232 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-11 00:08:09.268266646 +0000 2024-09-11T00:13:54.232 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-11 00:08:09.268266646 +0000 2024-09-11T00:13:54.232 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-11 00:08:09.268266646 +0000 2024-09-11T00:13:54.232 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T00:13:54.327 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-11T00:13:54.328 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-11T00:13:54.328 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000354025 s, 1.4 MB/s 2024-09-11T00:13:54.330 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T00:13:54.368 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-09-11T00:13:54.443 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T00:13:54.443 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:54.443 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-09-11T00:13:54.443 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:54.444 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:54.444 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-11 00:12:44.080168947 +0000 2024-09-11T00:13:54.444 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-11 00:08:09.731257334 +0000 2024-09-11T00:13:54.444 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-11 00:08:09.731257334 +0000 2024-09-11T00:13:54.444 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-11 00:08:09.731257334 +0000 2024-09-11T00:13:54.444 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T00:13:54.540 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-09-11T00:13:54.540 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-09-11T00:13:54.540 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000297704 s, 1.7 MB/s 2024-09-11T00:13:54.542 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T00:13:54.581 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-11T00:13:54.581 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T00:13:54.602 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T00:13:54.602 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-11 00:13:38.258659958 +0000 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-11 00:07:34.852626500 +0000 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-11 00:07:34.852626500 +0000 2024-09-11T00:13:54.661 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-11 00:07:34.852626500 +0000 2024-09-11T00:13:54.661 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T00:13:54.732 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-11T00:13:54.732 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-11T00:13:54.733 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000279756 s, 1.8 MB/s 2024-09-11T00:13:54.734 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T00:13:54.794 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-09-11T00:13:54.853 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T00:13:54.853 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:54.854 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-11T00:13:54.854 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:54.854 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:54.854 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-11 00:13:38.258659958 +0000 2024-09-11T00:13:54.854 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-11 00:07:35.177619834 +0000 2024-09-11T00:13:54.854 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-11 00:07:35.177619834 +0000 2024-09-11T00:13:54.854 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-11 00:07:35.177619834 +0000 2024-09-11T00:13:54.854 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T00:13:54.920 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-11T00:13:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-11T00:13:54.921 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000223316 s, 2.3 MB/s 2024-09-11T00:13:54.922 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T00:13:54.981 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-09-11T00:13:55.038 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T00:13:55.038 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:55.038 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T00:13:55.038 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:55.038 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:55.038 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-11 00:13:38.258659958 +0000 2024-09-11T00:13:55.038 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-11 00:07:35.513612943 +0000 2024-09-11T00:13:55.039 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-11 00:07:35.513612943 +0000 2024-09-11T00:13:55.039 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-11 00:07:35.513612943 +0000 2024-09-11T00:13:55.039 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T00:13:55.104 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-11T00:13:55.104 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-11T00:13:55.104 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00024579 s, 2.1 MB/s 2024-09-11T00:13:55.105 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T00:13:55.162 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-09-11T00:13:55.218 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-11 00:13:38.258659958 +0000 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-11 00:07:35.839606256 +0000 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-11 00:07:35.839606256 +0000 2024-09-11T00:13:55.219 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-11 00:07:35.839606256 +0000 2024-09-11T00:13:55.219 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T00:13:55.283 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-11T00:13:55.283 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-11T00:13:55.283 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000199248 s, 2.6 MB/s 2024-09-11T00:13:55.284 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T00:13:55.341 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-11T00:13:55.341 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T00:13:55.361 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T00:13:55.361 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-11T00:13:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T00:13:55.419 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:55.419 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T00:13:55.419 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:55.419 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:55.419 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-11 00:13:35.924374559 +0000 2024-09-11T00:13:55.419 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-11 00:07:34.132841141 +0000 2024-09-11T00:13:55.420 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-11 00:07:34.132841141 +0000 2024-09-11T00:13:55.420 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-11 00:07:34.132841141 +0000 2024-09-11T00:13:55.420 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T00:13:55.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:55 smithi117 ceph-mon[35525]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:55.489 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-11T00:13:55.489 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-11T00:13:55.490 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.00022915 s, 2.2 MB/s 2024-09-11T00:13:55.491 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T00:13:55.550 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-11T00:13:55.607 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T00:13:55.607 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:55.608 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-11T00:13:55.608 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:55.608 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:55.608 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-11 00:13:35.924374559 +0000 2024-09-11T00:13:55.608 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-11 00:07:34.467831841 +0000 2024-09-11T00:13:55.608 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-11 00:07:34.467831841 +0000 2024-09-11T00:13:55.608 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-11 00:07:34.467831841 +0000 2024-09-11T00:13:55.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T00:13:55.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:55 smithi103 ceph-mon[30578]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:55.671 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:55 smithi183 ceph-mon[35934]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:55.674 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-11T00:13:55.674 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-11T00:13:55.674 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000218194 s, 2.3 MB/s 2024-09-11T00:13:55.675 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T00:13:55.733 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-11T00:13:55.791 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T00:13:55.791 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:55.792 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-11T00:13:55.792 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:55.792 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:55.792 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-11 00:13:35.924374559 +0000 2024-09-11T00:13:55.792 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-11 00:07:34.814822208 +0000 2024-09-11T00:13:55.792 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-11 00:07:34.814822208 +0000 2024-09-11T00:13:55.792 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-11 00:07:34.814822208 +0000 2024-09-11T00:13:55.792 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T00:13:55.861 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-11T00:13:55.861 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-11T00:13:55.861 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000218004 s, 2.3 MB/s 2024-09-11T00:13:55.861 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T00:13:55.919 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-11T00:13:55.976 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T00:13:55.976 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T00:13:55.976 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-11T00:13:55.976 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:13:55.977 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T00:13:55.977 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-11 00:13:35.925374532 +0000 2024-09-11T00:13:55.977 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-11 00:07:35.144813047 +0000 2024-09-11T00:13:55.977 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-11 00:07:35.144813047 +0000 2024-09-11T00:13:55.977 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-11 00:07:35.144813047 +0000 2024-09-11T00:13:55.977 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T00:13:56.042 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-11T00:13:56.042 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-11T00:13:56.043 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000212805 s, 2.4 MB/s 2024-09-11T00:13:56.044 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T00:13:56.103 INFO:tasks.cephadm:Deploying osd.0 on smithi103 with /dev/vg_nvme/lv_4... 2024-09-11T00:13:56.103 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T00:13:56.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:56 smithi103 ceph-mon[30578]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:13:56.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:56 smithi117 ceph-mon[35525]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:56.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:56 smithi183 ceph-mon[35934]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:59.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:13:59 smithi103 ceph-mon[30578]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:59.588 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:13:59.622 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-09-11T00:13:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:13:59 smithi117 ceph-mon[35525]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:59.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:13:59 smithi183 ceph-mon[35934]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:13:59.986 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:14:00.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 -- 172.21.15.103:0/1608582247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c102a20 msgr2=0x7f5b3c102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:00.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1608582247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c102a20 0x7f5b3c102e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009930 tx=0x7f5b2c02a420 comp rx=0 tx=0).stop 2024-09-11T00:14:00.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 -- 172.21.15.103:0/1608582247 shutdown_connections 2024-09-11T00:14:00.592 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1608582247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b3c104540 0x7f5b3c10aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:00.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1608582247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c103b80 0x7f5b3c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:00.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1608582247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c102a20 0x7f5b3c102e20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:00.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 -- 172.21.15.103:0/1608582247 >> 172.21.15.103:0/1608582247 conn(0x7f5b3c0fe2f0 msgr2=0x7f5b3c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:00.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 -- 172.21.15.103:0/1608582247 shutdown_connections 2024-09-11T00:14:00.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.590+0000 7f5b41c00640 1 -- 172.21.15.103:0/1608582247 wait complete. 2024-09-11T00:14:00.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 Processor -- start 2024-09-11T00:14:00.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 -- start start 2024-09-11T00:14:00.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c102a20 0x7f5b3c076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:00.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c103b80 0x7f5b3c075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:00.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b3c104540 0x7f5b3c075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:00.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c10e680 con 0x7f5b3c103b80 2024-09-11T00:14:00.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c10e500 con 0x7f5b3c104540 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b41c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c10e800 con 0x7f5b3c102a20 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c103b80 0x7f5b3c075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c102a20 0x7f5b3c076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.591+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c103b80 0x7f5b3c075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42298/0 (socket says 172.21.15.103:42298) 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3a000640 1 -- 172.21.15.103:0/1096150553 learned_addr learned my addr 172.21.15.103:0/1096150553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c102a20 0x7f5b3c076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:46956/0 (socket says 172.21.15.103:46956) 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3b400640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b3c104540 0x7f5b3c075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3aa00640 1 -- 172.21.15.103:0/1096150553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b3c104540 msgr2=0x7f5b3c075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3aa00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b3c104540 0x7f5b3c075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3aa00640 1 -- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c103b80 msgr2=0x7f5b3c075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:00.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3aa00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c103b80 0x7f5b3c075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3aa00640 1 -- 172.21.15.103:0/1096150553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c076340 con 0x7f5b3c102a20 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3a000640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c103b80 0x7f5b3c075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b3aa00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c102a20 0x7f5b3c076cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005740 tx=0x7f5b2c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.592+0000 7f5b33400640 1 -- 172.21.15.103:0/1096150553 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c03f070 con 0x7f5b3c102a20 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.593+0000 7f5b33400640 1 -- 172.21.15.103:0/1096150553 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b2c03b460 con 0x7f5b3c102a20 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.593+0000 7f5b33400640 1 -- 172.21.15.103:0/1096150553 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c035070 con 0x7f5b3c102a20 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.593+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c1a2ec0 con 0x7f5b3c102a20 2024-09-11T00:14:00.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.593+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1a3260 con 0x7f5b3c102a20 2024-09-11T00:14:00.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.594+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c109da0 con 0x7f5b3c102a20 2024-09-11T00:14:00.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.596+0000 7f5b33400640 1 -- 172.21.15.103:0/1096150553 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f5b2c04b060 con 0x7f5b3c102a20 2024-09-11T00:14:00.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.597+0000 7f5b33400640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5b200b5250 0x7f5b200b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:00.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.598+0000 7f5b33400640 1 -- 172.21.15.103:0/1096150553 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5b2c1040a0 con 0x7f5b3c102a20 2024-09-11T00:14:00.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.598+0000 7f5b3a000640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5b200b5250 0x7f5b200b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:00.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.598+0000 7f5b3a000640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5b200b5250 0x7f5b200b7710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005fd0 tx=0x7f5b2800b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:00.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.599+0000 7f5b33400640 1 -- 172.21.15.103:0/1096150553 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5b2c0cc540 con 0x7f5b3c102a20 2024-09-11T00:14:00.795 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:00.793+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5b3c063590 con 0x7f5b200b5250 2024-09-11T00:14:01.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:01 smithi103 ceph-mon[30578]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:01.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:01.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:01.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:01.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:01 smithi117 ceph-mon[35525]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:01.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:01 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:01.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:01 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:01.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:01 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:01.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:01 smithi183 ceph-mon[35934]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:01.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:01 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:01.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:01 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:01.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:01 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:02.406 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:02 smithi103 ceph-mon[30578]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:02 smithi117 ceph-mon[35525]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:02 smithi183 ceph-mon[35934]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:03.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:03 smithi103 ceph-mon[30578]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:03.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:03 smithi117 ceph-mon[35525]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:03.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:03 smithi183 ceph-mon[35934]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:04.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:04 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2731552568' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b2d81a93-417a-401b-88db-d1512a1bb426"} : dispatch 2024-09-11T00:14:04.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:04 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2731552568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2d81a93-417a-401b-88db-d1512a1bb426"}]': finished 2024-09-11T00:14:04.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:04 smithi103 ceph-mon[30578]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T00:14:04.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:04 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:04.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:04 smithi103 ceph-mon[30578]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:04 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2731552568' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b2d81a93-417a-401b-88db-d1512a1bb426"} : dispatch 2024-09-11T00:14:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:04 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2731552568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2d81a93-417a-401b-88db-d1512a1bb426"}]': finished 2024-09-11T00:14:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:04 smithi117 ceph-mon[35525]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T00:14:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:04 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:04 smithi117 ceph-mon[35525]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:04 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2731552568' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b2d81a93-417a-401b-88db-d1512a1bb426"} : dispatch 2024-09-11T00:14:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:04 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2731552568' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b2d81a93-417a-401b-88db-d1512a1bb426"}]': finished 2024-09-11T00:14:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:04 smithi183 ceph-mon[35934]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T00:14:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:04 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:04.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:04 smithi183 ceph-mon[35934]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:05.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:05 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/926127097' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:05.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:05 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/926127097' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:05.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:05 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/926127097' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:06.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:06 smithi103 ceph-mon[30578]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:06.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:06 smithi117 ceph-mon[35525]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:06.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:06 smithi183 ceph-mon[35934]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:09.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:09 smithi103 ceph-mon[30578]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:09.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:09 smithi117 ceph-mon[35525]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:09.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:09 smithi183 ceph-mon[35934]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:11 smithi103 ceph-mon[30578]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:11.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:11 smithi117 ceph-mon[35525]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:11.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:11 smithi183 ceph-mon[35934]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:13.673 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:13 smithi103 ceph-mon[30578]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:13.673 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-11T00:14:13.673 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:13.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:13 smithi117 ceph-mon[35525]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:13.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-11T00:14:13.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:13.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:13 smithi183 ceph-mon[35934]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:13.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-11T00:14:13.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:14.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:14 smithi103 ceph-mon[30578]: Deploying daemon osd.0 on smithi103 2024-09-11T00:14:14.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:14 smithi103 ceph-mon[30578]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:14 smithi117 ceph-mon[35525]: Deploying daemon osd.0 on smithi103 2024-09-11T00:14:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:14 smithi117 ceph-mon[35525]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:14.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:14 smithi183 ceph-mon[35934]: Deploying daemon osd.0 on smithi103 2024-09-11T00:14:14.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:14 smithi183 ceph-mon[35934]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:17.357 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:17 smithi103 ceph-mon[30578]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:17.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:17 smithi117 ceph-mon[35525]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:17.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:17 smithi183 ceph-mon[35934]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:19.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:19 smithi103 ceph-mon[30578]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:19.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:19 smithi117 ceph-mon[35525]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:19.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:19 smithi183 ceph-mon[35934]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:21.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:21 smithi103 ceph-mon[30578]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:21 smithi117 ceph-mon[35525]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:21.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:21 smithi183 ceph-mon[35934]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:22.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:22 smithi103 ceph-mon[30578]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:22 smithi117 ceph-mon[35525]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:22 smithi183 ceph-mon[35934]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:24.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:24 smithi103 ceph-mon[30578]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:24 smithi117 ceph-mon[35525]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:25.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:25.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:24 smithi183 ceph-mon[35934]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:26.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.633+0000 7f5b33400640 1 -- 172.21.15.103:0/1096150553 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b3c063590 con 0x7f5b200b5250 2024-09-11T00:14:26.634 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 0 on host 'smithi103' 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5b200b5250 msgr2=0x7f5b200b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5b200b5250 0x7f5b200b7710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005fd0 tx=0x7f5b2800b4a0 comp rx=0 tx=0).stop 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c102a20 msgr2=0x7f5b3c076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c102a20 0x7f5b3c076cd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005740 tx=0x7f5b2c0042f0 comp rx=0 tx=0).stop 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 shutdown_connections 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5b200b5250 0x7f5b200b7710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b3c104540 0x7f5b3c075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b3c103b80 0x7f5b3c075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:26.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 --2- 172.21.15.103:0/1096150553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5b3c102a20 0x7f5b3c076cd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:26.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 >> 172.21.15.103:0/1096150553 conn(0x7f5b3c0fe2f0 msgr2=0x7f5b3c106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:26.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 shutdown_connections 2024-09-11T00:14:26.638 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:14:26.636+0000 7f5b41c00640 1 -- 172.21.15.103:0/1096150553 wait complete. 2024-09-11T00:14:27.436 DEBUG:teuthology.orchestra.run.smithi103:osd.0> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T00:14:27.440 INFO:tasks.cephadm:Deploying osd.1 on smithi117 with /dev/vg_nvme/lv_4... 2024-09-11T00:14:27.441 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T00:14:27.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:27 smithi103 ceph-mon[30578]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:27.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:27.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:27.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:27.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:27 smithi103 ceph-mon[30578]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:14:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:27 smithi117 ceph-mon[35525]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:27 smithi117 ceph-mon[35525]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:14:27.691 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:14:27.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:27 smithi183 ceph-mon[35934]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:27.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:27.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:27.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:27.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:27 smithi183 ceph-mon[35934]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-11T00:14:28.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:28 smithi103 ceph-mon[30578]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T00:14:28.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:28 smithi103 ceph-mon[30578]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T00:14:28.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:28 smithi103 ceph-mon[30578]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:14:28.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:28.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:28 smithi103 ceph-mon[30578]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:28.824 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:14:28 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0[46487]: 2024-09-11T00:14:28.649+0000 7f6529e00640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T00:14:28.824 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:14:28 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0[46487]: 2024-09-11T00:14:28.652+0000 7f6524e00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:14:28.835 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:28 smithi117 ceph-mon[35525]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T00:14:28.836 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:28 smithi117 ceph-mon[35525]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T00:14:28.836 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:28 smithi117 ceph-mon[35525]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:14:28.836 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:28.836 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:28 smithi117 ceph-mon[35525]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:29.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:28 smithi183 ceph-mon[35934]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T00:14:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:28 smithi183 ceph-mon[35934]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T00:14:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:28 smithi183 ceph-mon[35934]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:14:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:28 smithi183 ceph-mon[35934]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:29 smithi103 ceph-mon[30578]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:14:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:29 smithi103 ceph-mon[30578]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T00:14:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:29 smithi117 ceph-mon[35525]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:14:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:29 smithi117 ceph-mon[35525]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T00:14:29.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:29.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:29 smithi183 ceph-mon[35934]: from='osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:14:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:29 smithi183 ceph-mon[35934]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T00:14:30.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:30.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:30.052 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:14:30.082 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-09-11T00:14:30.345 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:14:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 -- 172.21.15.117:0/798746092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c102a70 msgr2=0x7ff26c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 --2- 172.21.15.117:0/798746092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c102a70 0x7ff26c102ef0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff258009930 tx=0x7ff25802a420 comp rx=0 tx=0).stop 2024-09-11T00:14:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 -- 172.21.15.117:0/798746092 shutdown_connections 2024-09-11T00:14:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 --2- 172.21.15.117:0/798746092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff26c103430 0x7ff26c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:30.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 --2- 172.21.15.117:0/798746092 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c102a70 0x7ff26c102ef0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:30.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 --2- 172.21.15.117:0/798746092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff26c101910 0x7ff26c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:30.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 -- 172.21.15.117:0/798746092 >> 172.21.15.117:0/798746092 conn(0x7ff26c0fd1e0 msgr2=0x7ff26c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:30.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.734+0000 7ff273244640 1 -- 172.21.15.117:0/798746092 shutdown_connections 2024-09-11T00:14:30.737 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.735+0000 7ff273244640 1 -- 172.21.15.117:0/798746092 wait complete. 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.735+0000 7ff273244640 1 Processor -- start 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.735+0000 7ff273244640 1 -- start start 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff273244640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff26c101910 0x7ff26c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff273244640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff26c102a70 0x7ff26c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff273244640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c103430 0x7ff26c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff273244640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c06b9f0 con 0x7ff26c101910 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff273244640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c06b870 con 0x7ff26c103430 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff273244640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c10e8c0 con 0x7ff26c102a70 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c103430 0x7ff26c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c103430 0x7ff26c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51404/0 (socket says 172.21.15.117:51404) 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 -- 172.21.15.117:0/3573523870 learned_addr learned my addr 172.21.15.117:0/3573523870 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-11T00:14:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff26bfff640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff26c102a70 0x7ff26c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 -- 172.21.15.117:0/3573523870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff26c102a70 msgr2=0x7ff26c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff26c102a70 0x7ff26c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 -- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff26c101910 msgr2=0x7ff26c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff270fe0640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff26c101910 0x7ff26c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff26c101910 0x7ff26c198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 -- 172.21.15.117:0/3573523870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c19a970 con 0x7ff26c103430 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff270fe0640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff26c101910 0x7ff26c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff2717e1640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c103430 0x7ff26c19a0d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff26400b970 tx=0x7ff26400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.736+0000 7ff269ffb640 1 -- 172.21.15.117:0/3573523870 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264019070 con 0x7ff26c103430 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.737+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c19ac60 con 0x7ff26c103430 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.737+0000 7ff269ffb640 1 -- 172.21.15.117:0/3573523870 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff264014440 con 0x7ff26c103430 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.737+0000 7ff269ffb640 1 -- 172.21.15.117:0/3573523870 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640075d0 con 0x7ff26c103430 2024-09-11T00:14:30.740 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.737+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c1a1340 con 0x7ff26c103430 2024-09-11T00:14:30.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.738+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c108c90 con 0x7ff26c103430 2024-09-11T00:14:30.741 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.739+0000 7ff269ffb640 1 -- 172.21.15.117:0/3573523870 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7ff26400cb40 con 0x7ff26c103430 2024-09-11T00:14:30.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.740+0000 7ff269ffb640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff25c0b52d0 0x7ff25c0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:30.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.740+0000 7ff269ffb640 1 -- 172.21.15.117:0/3573523870 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7ff2640e3d20 con 0x7ff26c103430 2024-09-11T00:14:30.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.740+0000 7ff270fe0640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff25c0b52d0 0x7ff25c0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:30.742 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.741+0000 7ff270fe0640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff25c0b52d0 0x7ff25c0b7790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2600058c0 tx=0x7ff260005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:30.745 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.743+0000 7ff269ffb640 1 -- 172.21.15.117:0/3573523870 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff2640ac000 con 0x7ff26c103430 2024-09-11T00:14:30.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:30 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:14:30.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:30 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:14:30.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:30 smithi103 ceph-mon[30578]: osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] boot 2024-09-11T00:14:30.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:30 smithi103 ceph-mon[30578]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T00:14:30.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:30 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:30.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:30 smithi103 ceph-mon[30578]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:30 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:14:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:30 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:14:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:30 smithi117 ceph-mon[35525]: osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] boot 2024-09-11T00:14:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:30 smithi117 ceph-mon[35525]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T00:14:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:30 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:30 smithi117 ceph-mon[35525]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:30.938 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:30.936+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff26c063590 con 0x7ff25c0b52d0 2024-09-11T00:14:31.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:30 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:14:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:30 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:14:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:30 smithi183 ceph-mon[35934]: osd.0 [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] boot 2024-09-11T00:14:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:30 smithi183 ceph-mon[35934]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T00:14:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:30 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-11T00:14:31.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:30 smithi183 ceph-mon[35934]: pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T00:14:31.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:31.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:31 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T00:14:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:31.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:14:31.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:31.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:31.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:31.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:31 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:32.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:32.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:31 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:32.685 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:32 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi103 2024-09-11T00:14:32.685 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:32 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi103 to 17089M 2024-09-11T00:14:32.686 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:32 smithi117 ceph-mon[35525]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:32.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:32 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi103 2024-09-11T00:14:32.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:32 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi103 to 17089M 2024-09-11T00:14:32.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:32 smithi103 ceph-mon[30578]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:33.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:32 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi103 2024-09-11T00:14:33.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:32 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi103 to 17089M 2024-09-11T00:14:33.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:32 smithi183 ceph-mon[35934]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:33.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:33 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/3180934964' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"} : dispatch 2024-09-11T00:14:33.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:33 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"} : dispatch 2024-09-11T00:14:33.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:33 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"}]': finished 2024-09-11T00:14:33.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:33 smithi103 ceph-mon[30578]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T00:14:33.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:33 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:33.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:33 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/3180934964' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"} : dispatch 2024-09-11T00:14:33.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:33 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"} : dispatch 2024-09-11T00:14:33.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:33 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"}]': finished 2024-09-11T00:14:33.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:33 smithi117 ceph-mon[35525]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T00:14:33.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:33 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:34.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:33 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/3180934964' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"} : dispatch 2024-09-11T00:14:34.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:33 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"} : dispatch 2024-09-11T00:14:34.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:33 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a35cffa-4c5a-4c39-851e-563d54f29b49"}]': finished 2024-09-11T00:14:34.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:33 smithi183 ceph-mon[35934]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T00:14:34.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:33 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:34.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:34 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/4193270371' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:34.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:34 smithi103 ceph-mon[30578]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:34 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/4193270371' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:34 smithi117 ceph-mon[35525]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:35.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:34 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/4193270371' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:35.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:34 smithi183 ceph-mon[35934]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:37.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:37 smithi103 ceph-mon[30578]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:37.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:37 smithi117 ceph-mon[35525]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:37.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:37 smithi183 ceph-mon[35934]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:39.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:39 smithi103 ceph-mon[30578]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:39.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:39 smithi117 ceph-mon[35525]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:39.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:39 smithi183 ceph-mon[35934]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:40.585 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:40 smithi117 ceph-mon[35525]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:40.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:40 smithi103 ceph-mon[30578]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:40.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:40 smithi183 ceph-mon[35934]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:41.560 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-11T00:14:41.561 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:41.561 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:41 smithi117 ceph-mon[35525]: Deploying daemon osd.1 on smithi117 2024-09-11T00:14:41.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-11T00:14:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:41 smithi103 ceph-mon[30578]: Deploying daemon osd.1 on smithi117 2024-09-11T00:14:41.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-11T00:14:41.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:41.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:41 smithi183 ceph-mon[35934]: Deploying daemon osd.1 on smithi117 2024-09-11T00:14:42.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:42 smithi103 ceph-mon[30578]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:42 smithi117 ceph-mon[35525]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:42 smithi183 ceph-mon[35934]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:45.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:45 smithi103 ceph-mon[30578]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:45.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:45 smithi117 ceph-mon[35525]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:45.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:45 smithi183 ceph-mon[35934]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:47.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:47 smithi103 ceph-mon[30578]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:47.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:47 smithi117 ceph-mon[35525]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:47.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:47 smithi183 ceph-mon[35934]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:48.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:48 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:48.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:48 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:48.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:48 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:48 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:48 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:48 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:48 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:48 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:48.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:48 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:49.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:49 smithi103 ceph-mon[30578]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:49.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:49 smithi117 ceph-mon[35525]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:49.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:49 smithi183 ceph-mon[35934]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:50.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:50.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:50.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:50.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:50.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:50.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:50 smithi103 ceph-mon[30578]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:50 smithi117 ceph-mon[35525]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:51.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:51.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:51.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:51.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:51.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:51.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:50 smithi183 ceph-mon[35934]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:51.432 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.430+0000 7ff269ffb640 1 -- 172.21.15.117:0/3573523870 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff26c063590 con 0x7ff25c0b52d0 2024-09-11T00:14:51.433 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 1 on host 'smithi117' 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff25c0b52d0 msgr2=0x7ff25c0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff25c0b52d0 0x7ff25c0b7790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2600058c0 tx=0x7ff260005d30 comp rx=0 tx=0).stop 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c103430 msgr2=0x7ff26c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c103430 0x7ff26c19a0d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff26400b970 tx=0x7ff26400be30 comp rx=0 tx=0).stop 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 shutdown_connections 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff25c0b52d0 0x7ff25c0b7790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff26c103430 0x7ff26c19a0d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff26c102a70 0x7ff26c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:51.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 --2- 172.21.15.117:0/3573523870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff26c101910 0x7ff26c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:51.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 >> 172.21.15.117:0/3573523870 conn(0x7ff26c0fd1e0 msgr2=0x7ff26c105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:51.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 shutdown_connections 2024-09-11T00:14:51.436 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:14:51.433+0000 7ff273244640 1 -- 172.21.15.117:0/3573523870 wait complete. 2024-09-11T00:14:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:51 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:51 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:51 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:51.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:51 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:51.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:51 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:51.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:51 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:52.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:51 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:14:52.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:51 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:52.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:51 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:52.046 DEBUG:teuthology.orchestra.run.smithi117:osd.1> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T00:14:52.048 INFO:tasks.cephadm:Deploying osd.2 on smithi183 with /dev/vg_nvme/lv_4... 2024-09-11T00:14:52.048 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T00:14:52.297 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:14:52.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:52 smithi183 ceph-mon[35934]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:52.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:52 smithi103 ceph-mon[30578]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:52.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:52 smithi117 ceph-mon[35525]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:53.748 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:14:53 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1[40006]: 2024-09-11T00:14:53.684+0000 7f8ac983a640 -1 osd.1 0 log_to_monitors true 2024-09-11T00:14:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:53 smithi183 ceph-mon[35934]: from='osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:14:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:53 smithi183 ceph-mon[35934]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:14:54.044 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:53 smithi117 ceph-mon[35525]: from='osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:14:54.045 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:53 smithi117 ceph-mon[35525]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:14:54.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:53 smithi103 ceph-mon[30578]: from='osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:14:54.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:53 smithi103 ceph-mon[30578]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-11T00:14:54.606 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:14:54.632 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-09-11T00:14:54.877 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:14:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:54 smithi117 ceph-mon[35525]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T00:14:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:54 smithi117 ceph-mon[35525]: from='osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:14:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:54 smithi117 ceph-mon[35525]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T00:14:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:54 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:54 smithi117 ceph-mon[35525]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:14:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:54 smithi117 ceph-mon[35525]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:54 smithi117 ceph-mon[35525]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:14:55.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:54 smithi183 ceph-mon[35934]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T00:14:55.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:54 smithi183 ceph-mon[35934]: from='osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:14:55.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:54 smithi183 ceph-mon[35934]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T00:14:55.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:54 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:55.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:54 smithi183 ceph-mon[35934]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:14:55.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:54 smithi183 ceph-mon[35934]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:55.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:54 smithi183 ceph-mon[35934]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:14:55.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:54 smithi103 ceph-mon[30578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T00:14:55.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:54 smithi103 ceph-mon[30578]: from='osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:14:55.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:54 smithi103 ceph-mon[30578]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T00:14:55.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:54 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:55.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:54 smithi103 ceph-mon[30578]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:14:55.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:54 smithi103 ceph-mon[30578]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:55.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:54 smithi103 ceph-mon[30578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:14:55.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 -- 172.21.15.183:0/3060797198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 msgr2=0x7f4830108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:55.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 --2- 172.21.15.183:0/3060797198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830108020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4824009900 tx=0x7f482402a410 comp rx=0 tx=0).stop 2024-09-11T00:14:55.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 -- 172.21.15.183:0/3060797198 shutdown_connections 2024-09-11T00:14:55.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 --2- 172.21.15.183:0/3060797198 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4830108760 0x7f483010ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:55.323 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 --2- 172.21.15.183:0/3060797198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830108020 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 --2- 172.21.15.183:0/3060797198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f483006c020 0x7f48301056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 -- 172.21.15.183:0/3060797198 >> 172.21.15.183:0/3060797198 conn(0x7f48300fd200 msgr2=0x7f48300ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 -- 172.21.15.183:0/3060797198 shutdown_connections 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.312+0000 7f4835ac9640 1 -- 172.21.15.183:0/3060797198 wait complete. 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 Processor -- start 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 -- start start 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f483006c020 0x7f4830102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4830108760 0x7f4830101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f483010e710 con 0x7f4830108760 2024-09-11T00:14:55.324 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f483010e590 con 0x7f483006c020 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f4835ac9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f483010e890 con 0x7f4830105c10 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f482effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f482effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830101670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:35846/0 (socket says 172.21.15.183:35846) 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f482f7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f483006c020 0x7f4830102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f482effd640 1 -- 172.21.15.183:0/1362288634 learned_addr learned my addr 172.21.15.183:0/1362288634 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.313+0000 7f482ffff640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4830108760 0x7f4830101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482effd640 1 -- 172.21.15.183:0/1362288634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f483006c020 msgr2=0x7f4830102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482effd640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f483006c020 0x7f4830102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482effd640 1 -- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4830108760 msgr2=0x7f4830101bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482effd640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4830108760 0x7f4830101bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482effd640 1 -- 172.21.15.183:0/1362288634 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483019cdd0 con 0x7f4830105c10 2024-09-11T00:14:55.325 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482ffff640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4830108760 0x7f4830101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482f7fe640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f483006c020 0x7f4830102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482effd640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830101670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4824005940 tx=0x7f4824004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482cff9640 1 -- 172.21.15.183:0/1362288634 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482403f030 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482cff9640 1 -- 172.21.15.183:0/1362288634 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4824003cb0 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483019d000 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f482cff9640 1 -- 172.21.15.183:0/1362288634 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824035070 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.314+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483019d4e0 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.315+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48300fecf0 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.317+0000 7f482cff9640 1 -- 172.21.15.183:0/1362288634 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f482403b460 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.318+0000 7f482cff9640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f48180b5250 0x7f48180b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.318+0000 7f482cff9640 1 -- 172.21.15.183:0/1362288634 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f48241050d0 con 0x7f4830105c10 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.318+0000 7f482f7fe640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f48180b5250 0x7f48180b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.319+0000 7f482f7fe640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f48180b5250 0x7f48180b7710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4820005d00 tx=0x7f482000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:14:55.326 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.321+0000 7f482cff9640 1 -- 172.21.15.183:0/1362288634 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f48240cd900 con 0x7f4830105c10 2024-09-11T00:14:55.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:14:55.507+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4830102160 con 0x7f48180b5250 2024-09-11T00:14:55.687 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1[40006]: 2024-09-11T00:14:55.375+0000 7f8ac57bd640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T00:14:55.687 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1[40006]: 2024-09-11T00:14:55.379+0000 7f8abd5dc640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:14:56.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T00:14:56.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.015 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi117 2024-09-11T00:14:56.016 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.016 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.016 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:14:56.016 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-11T00:14:56.016 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.016 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:56.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:56.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='osd.1 ' entity='osd.1' 2024-09-11T00:14:56.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:56.017 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:56.018 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:56.018 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T00:14:56.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi117 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='osd.1 ' entity='osd.1' 2024-09-11T00:14:56.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:56.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:56.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:56.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T00:14:56.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:56.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi117 2024-09-11T00:14:56.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:14:56.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi117 to 17088M 2024-09-11T00:14:56.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='osd.1 ' entity='osd.1' 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:14:56.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:57.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:56 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:14:57.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:56 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:14:57.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:56 smithi183 ceph-mon[35934]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:57.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:56 smithi183 ceph-mon[35934]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:57.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:56 smithi183 ceph-mon[35934]: osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] boot 2024-09-11T00:14:57.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:56 smithi183 ceph-mon[35934]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T00:14:57.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:56 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:57.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:56 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:14:57.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:56 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:14:57.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:56 smithi103 ceph-mon[30578]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:57.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:56 smithi103 ceph-mon[30578]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:57.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:56 smithi103 ceph-mon[30578]: osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] boot 2024-09-11T00:14:57.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:56 smithi103 ceph-mon[30578]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T00:14:57.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:56 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:57.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:56 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:14:57.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:56 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:14:57.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:56 smithi117 ceph-mon[35525]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:14:57.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:56 smithi117 ceph-mon[35525]: pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-11T00:14:57.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:56 smithi117 ceph-mon[35525]: osd.1 [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] boot 2024-09-11T00:14:57.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:56 smithi117 ceph-mon[35525]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T00:14:57.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:56 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-11T00:14:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T00:14:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/20085599' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"} : dispatch 2024-09-11T00:14:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"} : dispatch 2024-09-11T00:14:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"}]': finished 2024-09-11T00:14:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T00:14:58.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:14:58.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:14:58.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:14:58 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/2329534751' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T00:14:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/20085599' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"} : dispatch 2024-09-11T00:14:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"} : dispatch 2024-09-11T00:14:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"}]': finished 2024-09-11T00:14:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T00:14:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:14:58.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:14:58.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:14:58 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/2329534751' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:14:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T00:14:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/20085599' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"} : dispatch 2024-09-11T00:14:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"} : dispatch 2024-09-11T00:14:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fca0072a-688a-4c07-90b9-bd4bb84aa15f"}]': finished 2024-09-11T00:14:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T00:14:58.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:14:58.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:14:58.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:14:58 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/2329534751' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:15:01.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:01 smithi103 ceph-mon[30578]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:01.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:01 smithi117 ceph-mon[35525]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:01.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:01 smithi183 ceph-mon[35934]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:03.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:03 smithi183 ceph-mon[35934]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:03.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:03 smithi103 ceph-mon[30578]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:03.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:03 smithi117 ceph-mon[35525]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:04.617 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:04 smithi183 ceph-mon[35934]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:04.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:04 smithi103 ceph-mon[30578]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:04 smithi117 ceph-mon[35525]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:06.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:05 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-11T00:15:06.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:05 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:06.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:05 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-11T00:15:06.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:05 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:06.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:05 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-11T00:15:06.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:05 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:07.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:06 smithi183 ceph-mon[35934]: Deploying daemon osd.2 on smithi183 2024-09-11T00:15:07.137 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:06 smithi183 ceph-mon[35934]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:07.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:06 smithi103 ceph-mon[30578]: Deploying daemon osd.2 on smithi183 2024-09-11T00:15:07.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:06 smithi103 ceph-mon[30578]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:07.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:06 smithi117 ceph-mon[35525]: Deploying daemon osd.2 on smithi183 2024-09-11T00:15:07.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:06 smithi117 ceph-mon[35525]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:09.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:09 smithi103 ceph-mon[30578]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:09.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:09 smithi117 ceph-mon[35525]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:09.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:09 smithi183 ceph-mon[35934]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:11.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:11 smithi183 ceph-mon[35934]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:11.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:11 smithi103 ceph-mon[30578]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:11.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:11 smithi117 ceph-mon[35525]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:12.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:12.702 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:12.703 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:13.661 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:13 smithi183 ceph-mon[35934]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:13.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:13 smithi103 ceph-mon[30578]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:13.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:13 smithi117 ceph-mon[35525]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:14.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:14 smithi103 ceph-mon[30578]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:14 smithi117 ceph-mon[35525]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:14.692 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:14 smithi183 ceph-mon[35934]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:15.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:15.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:15.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:15.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:15.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.505 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.503+0000 7f482cff9640 1 -- 172.21.15.183:0/1362288634 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4830102160 con 0x7f48180b5250 2024-09-11T00:15:16.505 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 2 on host 'smithi183' 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f48180b5250 msgr2=0x7f48180b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f48180b5250 0x7f48180b7710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4820005d00 tx=0x7f482000b410 comp rx=0 tx=0).stop 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 msgr2=0x7f4830101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830101670 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4824005940 tx=0x7f4824004430 comp rx=0 tx=0).stop 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 shutdown_connections 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f48180b5250 0x7f48180b7710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4830108760 0x7f4830101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4830105c10 0x7f4830101670 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 --2- 172.21.15.183:0/1362288634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f483006c020 0x7f4830102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:16.508 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 >> 172.21.15.183:0/1362288634 conn(0x7f48300fd200 msgr2=0x7f4830103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:16.509 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 shutdown_connections 2024-09-11T00:15:16.509 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:15:16.506+0000 7f4835ac9640 1 -- 172.21.15.183:0/1362288634 wait complete. 2024-09-11T00:15:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:16 smithi183 ceph-mon[35934]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:16.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:16.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:16 smithi103 ceph-mon[30578]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:16.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:16.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:16 smithi117 ceph-mon[35525]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:17.170 DEBUG:teuthology.orchestra.run.smithi183:osd.2> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T00:15:17.172 INFO:tasks.cephadm:Deploying osd.3 on smithi103 with /dev/vg_nvme/lv_3... 2024-09-11T00:15:17.172 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T00:15:17.540 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:15:18.531 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:18 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2[40887]: 2024-09-11T00:15:18.390+0000 7f1f0a4e0640 -1 osd.2 0 log_to_monitors true 2024-09-11T00:15:19.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:19 smithi183 ceph-mon[35934]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:19.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:19 smithi183 ceph-mon[35934]: from='osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:15:19.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:19 smithi183 ceph-mon[35934]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:15:19.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:19 smithi103 ceph-mon[30578]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:19.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:19 smithi103 ceph-mon[30578]: from='osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:15:19.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:19 smithi103 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:15:19.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:19 smithi117 ceph-mon[35525]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:19.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:19 smithi117 ceph-mon[35525]: from='osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:15:19.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:19 smithi117 ceph-mon[35525]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-11T00:15:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T00:15:20.740 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T00:15:20.740 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:20.740 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:15:20.740 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi183 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:20 smithi103 ceph-mon[30578]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi183 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.744 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.745 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:20.745 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-11T00:15:20.745 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.745 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:20.745 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:20.745 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.745 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:20 smithi117 ceph-mon[35525]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T00:15:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi183 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:20 smithi183 ceph-mon[35934]: pgmap v93: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:20.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:15:20.968 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-09-11T00:15:21.339 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:15:21.364 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2[40887]: 2024-09-11T00:15:21.040+0000 7f1f06463640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T00:15:21.364 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2[40887]: 2024-09-11T00:15:21.044+0000 7f1efe282640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:15:21.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:21 smithi103 ceph-mon[30578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:15:21.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:21 smithi103 ceph-mon[30578]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T00:15:21.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:21.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:21.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:21 smithi103 ceph-mon[30578]: from='osd.2 ' entity='osd.2' 2024-09-11T00:15:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:21 smithi117 ceph-mon[35525]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:15:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:21 smithi117 ceph-mon[35525]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T00:15:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:21 smithi117 ceph-mon[35525]: from='osd.2 ' entity='osd.2' 2024-09-11T00:15:21.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:21 smithi183 ceph-mon[35934]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:15:21.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:21 smithi183 ceph-mon[35934]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T00:15:21.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:21.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:21.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:21 smithi183 ceph-mon[35934]: from='osd.2 ' entity='osd.2' 2024-09-11T00:15:21.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 -- 172.21.15.103:0/2206991020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 msgr2=0x7f30601033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 --2- 172.21.15.103:0/2206991020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f30601033f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3050009900 tx=0x7f305002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 -- 172.21.15.103:0/2206991020 shutdown_connections 2024-09-11T00:15:21.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 --2- 172.21.15.103:0/2206991020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3060106530 0x7f30601069e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 --2- 172.21.15.103:0/2206991020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3060103980 0x7f3060105e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 --2- 172.21.15.103:0/2206991020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f30601033f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 -- 172.21.15.103:0/2206991020 >> 172.21.15.103:0/2206991020 conn(0x7f30600fd1f0 msgr2=0x7f30600ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 -- 172.21.15.103:0/2206991020 shutdown_connections 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.967+0000 7f3066600640 1 -- 172.21.15.103:0/2206991020 wait complete. 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 Processor -- start 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 -- start start 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f306019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3060103980 0x7f306019fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3060106530 0x7f30601a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3060114d00 con 0x7f306006c160 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3060114b80 con 0x7f3060103980 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.968+0000 7f3066600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3060114e80 con 0x7f3060106530 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f306019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f306019f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45676/0 (socket says 172.21.15.103:45676) 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 -- 172.21.15.103:0/2512922038 learned_addr learned my addr 172.21.15.103:0/2512922038 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305ea00640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3060103980 0x7f306019fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305fe00640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3060106530 0x7f30601a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 -- 172.21.15.103:0/2512922038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3060106530 msgr2=0x7f30601a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3060106530 0x7f30601a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 -- 172.21.15.103:0/2512922038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3060103980 msgr2=0x7f306019fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3060103980 0x7f306019fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 -- 172.21.15.103:0/2512922038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601a0f70 con 0x7f306006c160 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305fe00640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3060106530 0x7f30601a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.969+0000 7f305f400640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f306019f440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3050005ab0 tx=0x7f3050003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.970+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305003f070 con 0x7f306006c160 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.970+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f305003b460 con 0x7f306006c160 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.970+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050035030 con 0x7f306006c160 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.970+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30601a1200 con 0x7f306006c160 2024-09-11T00:15:21.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.970+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30601a7940 con 0x7f306006c160 2024-09-11T00:15:21.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.971+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30601069e0 con 0x7f306006c160 2024-09-11T00:15:21.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.973+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 153195+0+0 (secure 0 0 0) 0x7f305003b8d0 con 0x7f306006c160 2024-09-11T00:15:21.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.974+0000 7f3057e00640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f30440b52d0 0x7f30440b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:21.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.974+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f30501053e0 con 0x7f306006c160 2024-09-11T00:15:21.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.975+0000 7f305ea00640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f30440b52d0 0x7f30440b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:21.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.975+0000 7f305ea00640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f30440b52d0 0x7f30440b7790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f304c005fd0 tx=0x7f304c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:21.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:21.976+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f30500cd950 con 0x7f306006c160 2024-09-11T00:15:22.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:22.177+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3060063650 con 0x7f30440b52d0 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] boot 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:22 smithi103 ceph-mon[30578]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:15:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:15:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] boot 2024-09-11T00:15:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T00:15:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:15:22.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:15:22.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:15:22.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:22.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:22 smithi117 ceph-mon[35525]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: osd.2 [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] boot 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: from='client.14394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:22 smithi183 ceph-mon[35934]: pgmap v96: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-11T00:15:23.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:15:23.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:15:23.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:15:24.629 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T00:15:24.629 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 ceph-mon[35934]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T00:15:24.629 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:15:24.629 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 ceph-mon[35934]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T00:15:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 ceph-mon[30578]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T00:15:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:15:24.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 ceph-mon[30578]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51156]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51156]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51160]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51160]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:24.649 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51160]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T00:15:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 ceph-mon[35525]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T00:15:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:15:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 ceph-mon[35525]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:24.687 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43715]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:15:24.687 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43715]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:24.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43715]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:24.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43715]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:24.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43718]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:15:24.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43718]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:24.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43718]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:24.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43718]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:24.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51165]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:15:24.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51165]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51165]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51165]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51168]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:15:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51168]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51168]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:24.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:24 smithi103 sudo[51168]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:25.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44669]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:15:25.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44669]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44669]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44669]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44672]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:15:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44672]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44672]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:25.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44672]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:25.031 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:15:25.031 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:25.031 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44663]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:25.031 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44663]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:25.031 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44666]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:15:25.032 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44666]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:25.032 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44666]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:25.032 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:15:24 smithi183 sudo[44666]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:25.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43721]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T00:15:25.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:25.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43721]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:25.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43721]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:25.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43724]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T00:15:25.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T00:15:25.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43724]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T00:15:25.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:24 smithi117 sudo[43724]: pam_unix(sudo:session): session closed for user root 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:25 smithi103 ceph-mon[30578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:15:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:15:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T00:15:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:25 smithi117 ceph-mon[35525]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-11T00:15:25.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:25 smithi183 ceph-mon[35934]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T00:15:26.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:26.483+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f30500c94c0 con 0x7f306006c160 2024-09-11T00:15:26.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:26 smithi103 ceph-mon[30578]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T00:15:26.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:26 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/166260471' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39a7f76f-3edd-4144-b4c4-a3e2ca84001d"} : dispatch 2024-09-11T00:15:26.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:26 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/166260471' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39a7f76f-3edd-4144-b4c4-a3e2ca84001d"}]': finished 2024-09-11T00:15:26.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:26 smithi103 ceph-mon[30578]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T00:15:26.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:26.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:26 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3177872981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:15:26.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:26 smithi103 ceph-mon[30578]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:26 smithi117 ceph-mon[35525]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T00:15:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:26 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/166260471' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39a7f76f-3edd-4144-b4c4-a3e2ca84001d"} : dispatch 2024-09-11T00:15:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:26 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/166260471' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39a7f76f-3edd-4144-b4c4-a3e2ca84001d"}]': finished 2024-09-11T00:15:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:26 smithi117 ceph-mon[35525]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T00:15:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:26 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:26.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:26 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3177872981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:15:26.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:26 smithi117 ceph-mon[35525]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:26 smithi183 ceph-mon[35934]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T00:15:26.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:26 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/166260471' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "39a7f76f-3edd-4144-b4c4-a3e2ca84001d"} : dispatch 2024-09-11T00:15:26.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:26 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/166260471' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "39a7f76f-3edd-4144-b4c4-a3e2ca84001d"}]': finished 2024-09-11T00:15:26.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:26 smithi183 ceph-mon[35934]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T00:15:26.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:26 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:26.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:26 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3177872981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:15:26.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:26 smithi183 ceph-mon[35934]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:27 smithi103 ceph-mon[30578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:15:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:27 smithi103 ceph-mon[30578]: Cluster is now healthy 2024-09-11T00:15:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:27 smithi103 ceph-mon[30578]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-11T00:15:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:27 smithi117 ceph-mon[35525]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:15:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:27 smithi117 ceph-mon[35525]: Cluster is now healthy 2024-09-11T00:15:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:27 smithi117 ceph-mon[35525]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-11T00:15:27.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:27 smithi183 ceph-mon[35934]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:15:27.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:27 smithi183 ceph-mon[35934]: Cluster is now healthy 2024-09-11T00:15:27.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:27 smithi183 ceph-mon[35934]: mgrmap e16: x(active, since 3m), standbys: z, y 2024-09-11T00:15:28.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:28 smithi103 ceph-mon[30578]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:28 smithi117 ceph-mon[35525]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:28 smithi183 ceph-mon[35934]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:31.614 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:31 smithi103 ceph-mon[30578]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:31.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:31 smithi117 ceph-mon[35525]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:31.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:31 smithi183 ceph-mon[35934]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:33.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:33 smithi103 ceph-mon[30578]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:33 smithi117 ceph-mon[35525]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:33.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:33 smithi183 ceph-mon[35934]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:34.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:34 smithi103 ceph-mon[30578]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:34 smithi117 ceph-mon[35525]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:34 smithi183 ceph-mon[35934]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:35.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-11T00:15:35.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:35.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:35 smithi103 ceph-mon[30578]: Deploying daemon osd.3 on smithi103 2024-09-11T00:15:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-11T00:15:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:35 smithi117 ceph-mon[35525]: Deploying daemon osd.3 on smithi103 2024-09-11T00:15:35.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-11T00:15:35.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:35.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:35 smithi183 ceph-mon[35934]: Deploying daemon osd.3 on smithi103 2024-09-11T00:15:36.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:36 smithi103 ceph-mon[30578]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:36 smithi117 ceph-mon[35525]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:36 smithi183 ceph-mon[35934]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:39.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:39 smithi103 ceph-mon[30578]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:39.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:39 smithi117 ceph-mon[35525]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:39.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:39 smithi183 ceph-mon[35934]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:41 smithi103 ceph-mon[30578]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:41 smithi117 ceph-mon[35525]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:41.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:41 smithi183 ceph-mon[35934]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:42.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:42 smithi103 ceph-mon[30578]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:42 smithi117 ceph-mon[35525]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:42 smithi183 ceph-mon[35934]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:43.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:43.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:43.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:43.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:43.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:43.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:43.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:43.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:43.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:44 smithi103 ceph-mon[30578]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:44.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:44 smithi117 ceph-mon[35525]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:44 smithi183 ceph-mon[35934]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:47.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:47.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:47.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:47 smithi103 ceph-mon[30578]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:47 smithi117 ceph-mon[35525]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:47.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:47.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:47.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:47.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:47 smithi183 ceph-mon[35934]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:48.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.381+0000 7f3057e00640 1 -- 172.21.15.103:0/2512922038 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3060063650 con 0x7f30440b52d0 2024-09-11T00:15:48.384 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 3 on host 'smithi103' 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.383+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f30440b52d0 msgr2=0x7f30440b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.383+0000 7f3066600640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f30440b52d0 0x7f30440b7790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f304c005fd0 tx=0x7f304c00b410 comp rx=0 tx=0).stop 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.383+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 msgr2=0x7f306019f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.383+0000 7f3066600640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f306019f440 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3050005ab0 tx=0x7f3050003cd0 comp rx=0 tx=0).stop 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.383+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 shutdown_connections 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.384+0000 7f3066600640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f30440b52d0 0x7f30440b7790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.384+0000 7f3066600640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3060106530 0x7f30601a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.384+0000 7f3066600640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3060103980 0x7f306019fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.384+0000 7f3066600640 1 --2- 172.21.15.103:0/2512922038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f306006c160 0x7f306019f440 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:48.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.384+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 >> 172.21.15.103:0/2512922038 conn(0x7f30600fd1f0 msgr2=0x7f30601048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:48.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.384+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 shutdown_connections 2024-09-11T00:15:48.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:15:48.384+0000 7f3066600640 1 -- 172.21.15.103:0/2512922038 wait complete. 2024-09-11T00:15:49.208 DEBUG:teuthology.orchestra.run.smithi103:osd.3> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T00:15:49.213 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/vg_nvme/lv_3... 2024-09-11T00:15:49.213 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T00:15:49.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:49 smithi103 ceph-mon[30578]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:49.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:49.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:49.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:49.467 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:15:49.587 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:49 smithi117 ceph-mon[35525]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:49.588 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:49 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:49.588 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:49 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:49.588 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:49 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:49.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:49 smithi183 ceph-mon[35934]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:49.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:49 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:15:49.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:49 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:49.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:49 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:49.790 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:15:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:15:49.518+0000 7f75d5523640 -1 osd.3 0 log_to_monitors true 2024-09-11T00:15:50.619 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:50 smithi117 ceph-mon[35525]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-11T00:15:50.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:50 smithi103 ceph-mon[30578]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-11T00:15:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:50 smithi183 ceph-mon[35934]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-11T00:15:51.381 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:51 smithi103 ceph-mon[30578]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:51.524 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:51 smithi103 ceph-mon[30578]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T00:15:51.524 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:51 smithi103 ceph-mon[30578]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T00:15:51.524 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:51 smithi103 ceph-mon[30578]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:15:51.525 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:51 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:51.620 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:51 smithi117 ceph-mon[35525]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:51.620 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:51 smithi117 ceph-mon[35525]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T00:15:51.621 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:51 smithi117 ceph-mon[35525]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T00:15:51.621 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:51 smithi117 ceph-mon[35525]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:15:51.621 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:51 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:51.669 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:15:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:15:51.376+0000 7f75cf600640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T00:15:51.669 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:15:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:15:51.382+0000 7f75ca600640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:15:51.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:51 smithi183 ceph-mon[35934]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:51.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:51 smithi183 ceph-mon[35934]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T00:15:51.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:51 smithi183 ceph-mon[35934]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T00:15:51.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:51 smithi183 ceph-mon[35934]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:15:51.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:51 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:51.865 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:15:51.891 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-09-11T00:15:52.132 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:15:52.549 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:52 smithi117 ceph-mon[35525]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:15:52.549 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:52 smithi117 ceph-mon[35525]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T00:15:52.550 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:52 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:52.550 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:52 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:52.550 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:52 smithi117 ceph-mon[35525]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 -- 172.21.15.117:0/3652295301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60104560 msgr2=0x7f4e6010aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 --2- 172.21.15.117:0/3652295301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60104560 0x7f4e6010aee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009cd0 tx=0x7f4e5802a8e0 comp rx=0 tx=0).stop 2024-09-11T00:15:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 -- 172.21.15.117:0/3652295301 shutdown_connections 2024-09-11T00:15:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 --2- 172.21.15.117:0/3652295301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60104560 0x7f4e6010aee0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:52.633 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 --2- 172.21.15.117:0/3652295301 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60103ba0 0x7f4e60104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 --2- 172.21.15.117:0/3652295301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60102a40 0x7f4e60102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 -- 172.21.15.117:0/3652295301 >> 172.21.15.117:0/3652295301 conn(0x7f4e600fe2f0 msgr2=0x7f4e60100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 -- 172.21.15.117:0/3652295301 shutdown_connections 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.628+0000 7f4e68698640 1 -- 172.21.15.117:0/3652295301 wait complete. 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 Processor -- start 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 -- start start 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60102a40 0x7f4e60198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60103ba0 0x7f4e601994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60104560 0x7f4e6019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6010e720 con 0x7f4e60104560 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6010e5a0 con 0x7f4e60102a40 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e68698640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6010e8a0 con 0x7f4e60103ba0 2024-09-11T00:15:52.634 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60102a40 0x7f4e60198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60102a40 0x7f4e60198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57306/0 (socket says 172.21.15.117:57306) 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 -- 172.21.15.117:0/851895810 learned_addr learned my addr 172.21.15.117:0/851895810 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 -- 172.21.15.117:0/851895810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60103ba0 msgr2=0x7f4e601994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e65c33640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60103ba0 0x7f4e601994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60103ba0 0x7f4e601994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 -- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60104560 msgr2=0x7f4e6019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60104560 0x7f4e6019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e66434640 1 -- 172.21.15.117:0/851895810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6019a9c0 con 0x7f4e60102a40 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.629+0000 7f4e65c33640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60103ba0 0x7f4e601994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.630+0000 7f4e66434640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60102a40 0x7f4e60198e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400b4a0 tx=0x7f4e5400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.630+0000 7f4e537fe640 1 -- 172.21.15.117:0/851895810 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54014070 con 0x7f4e60102a40 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.630+0000 7f4e537fe640 1 -- 172.21.15.117:0/851895810 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e54010ac0 con 0x7f4e60102a40 2024-09-11T00:15:52.635 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.630+0000 7f4e537fe640 1 -- 172.21.15.117:0/851895810 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54017c30 con 0x7f4e60102a40 2024-09-11T00:15:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.630+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6019acb0 con 0x7f4e60102a40 2024-09-11T00:15:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.630+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e601a1390 con 0x7f4e60102a40 2024-09-11T00:15:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.631+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e60109d60 con 0x7f4e60102a40 2024-09-11T00:15:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.634+0000 7f4e537fe640 1 -- 172.21.15.117:0/851895810 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f4e54010610 con 0x7f4e60102a40 2024-09-11T00:15:52.636 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.634+0000 7f4e537fe640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e4c0b5430 0x7f4e4c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:15:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.634+0000 7f4e537fe640 1 -- 172.21.15.117:0/851895810 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7f4e540e42c0 con 0x7f4e60102a40 2024-09-11T00:15:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.634+0000 7f4e65c33640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e4c0b5430 0x7f4e4c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:15:52.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.635+0000 7f4e65c33640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e4c0b5430 0x7f4e4c0b78f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0058c0 tx=0x7f4e5c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:15:52.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.637+0000 7f4e537fe640 1 -- 172.21.15.117:0/851895810 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4e540ac400 con 0x7f4e60102a40 2024-09-11T00:15:52.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:52 smithi103 ceph-mon[30578]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:15:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:52 smithi103 ceph-mon[30578]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T00:15:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:52 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:52 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:52 smithi103 ceph-mon[30578]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:52 smithi183 ceph-mon[35934]: from='osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:15:52.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:52 smithi183 ceph-mon[35934]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T00:15:52.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:52 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:52.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:52 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:52.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:52 smithi183 ceph-mon[35934]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-11T00:15:52.825 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:15:52.823+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4e60063590 con 0x7f4e4c0b5430 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] boot 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T00:15:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi103 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.502 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:15:53.502 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] boot 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi103 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:53.503 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:53.504 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-11T00:15:53.504 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.504 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:53.504 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:53.504 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: osd.3 [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] boot 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T00:15:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi103 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi103 to 8544M 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:15:53.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:15:55.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:55 smithi117 ceph-mon[35525]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T00:15:55.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:55 smithi117 ceph-mon[35525]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:15:55.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:55 smithi183 ceph-mon[35934]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T00:15:55.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:55 smithi183 ceph-mon[35934]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:15:55.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:55 smithi103 ceph-mon[30578]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T00:15:55.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:55 smithi103 ceph-mon[30578]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:15:56.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:56 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/3235248860' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"} : dispatch 2024-09-11T00:15:56.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:56 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"} : dispatch 2024-09-11T00:15:56.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:56 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"}]': finished 2024-09-11T00:15:56.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:56 smithi183 ceph-mon[35934]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T00:15:56.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:56 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:15:56.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:56 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/2787447781' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:15:56.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:56 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/3235248860' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"} : dispatch 2024-09-11T00:15:56.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:56 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"} : dispatch 2024-09-11T00:15:56.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:56 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"}]': finished 2024-09-11T00:15:56.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:56 smithi103 ceph-mon[30578]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T00:15:56.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:56 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:15:56.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:56 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/2787447781' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:15:56.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:56 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/3235248860' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"} : dispatch 2024-09-11T00:15:56.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:56 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"} : dispatch 2024-09-11T00:15:56.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:56 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f479678-1b4e-4d7b-9c3e-d2033337c993"}]': finished 2024-09-11T00:15:56.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:56 smithi117 ceph-mon[35525]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T00:15:56.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:56 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:15:56.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:56 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/2787447781' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:15:57.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:57 smithi183 ceph-mon[35934]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:15:57.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:57 smithi103 ceph-mon[30578]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:15:57.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:57 smithi117 ceph-mon[35525]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:15:59.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:15:59 smithi103 ceph-mon[30578]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T00:15:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:15:59 smithi117 ceph-mon[35525]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T00:15:59.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:15:59 smithi183 ceph-mon[35934]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-11T00:16:00.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:00 smithi103 ceph-mon[30578]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T00:16:00.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:00 smithi117 ceph-mon[35525]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T00:16:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:00 smithi183 ceph-mon[35934]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-11T00:16:03.518 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:03 smithi117 ceph-mon[35525]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T00:16:03.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:03 smithi103 ceph-mon[30578]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T00:16:03.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:03 smithi183 ceph-mon[35934]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T00:16:04.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:04 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-11T00:16:04.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:04 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:04 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-11T00:16:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:04 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:04 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-11T00:16:04.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:04 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:05.561 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:05 smithi117 ceph-mon[35525]: Deploying daemon osd.4 on smithi117 2024-09-11T00:16:05.561 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:05 smithi117 ceph-mon[35525]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:16:05.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:05 smithi103 ceph-mon[30578]: Deploying daemon osd.4 on smithi117 2024-09-11T00:16:05.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:05 smithi103 ceph-mon[30578]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:16:05.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:05 smithi183 ceph-mon[35934]: Deploying daemon osd.4 on smithi117 2024-09-11T00:16:05.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:05 smithi183 ceph-mon[35934]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:16:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:06 smithi103 ceph-mon[30578]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T00:16:06.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:06 smithi117 ceph-mon[35525]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T00:16:06.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:06 smithi183 ceph-mon[35934]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-11T00:16:09.552 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:09 smithi117 ceph-mon[35525]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:16:09.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:09 smithi103 ceph-mon[30578]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:16:09.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:09 smithi183 ceph-mon[35934]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:16:10.630 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:10.630 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:10.630 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:10.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:10.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:10.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:10.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:11.644 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:11 smithi117 ceph-mon[35525]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:11.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:11 smithi103 ceph-mon[30578]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:11.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:11 smithi183 ceph-mon[35934]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:12.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:12 smithi103 ceph-mon[30578]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:12 smithi117 ceph-mon[35525]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:12.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:12 smithi183 ceph-mon[35934]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:13.494 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2024-09-11T00:16:13.494 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.492+0000 7f4e537fe640 1 -- 172.21.15.117:0/851895810 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4e60063590 con 0x7f4e4c0b5430 2024-09-11T00:16:13.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e4c0b5430 msgr2=0x7f4e4c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:13.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e4c0b5430 0x7f4e4c0b78f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0058c0 tx=0x7f4e5c005d30 comp rx=0 tx=0).stop 2024-09-11T00:16:13.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60102a40 msgr2=0x7f4e60198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:13.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60102a40 0x7f4e60198e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400b4a0 tx=0x7f4e5400b960 comp rx=0 tx=0).stop 2024-09-11T00:16:13.496 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 shutdown_connections 2024-09-11T00:16:13.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e4c0b5430 0x7f4e4c0b78f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:13.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e60104560 0x7f4e6019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:13.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60103ba0 0x7f4e601994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:13.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 --2- 172.21.15.117:0/851895810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e60102a40 0x7f4e60198e90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:13.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.494+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 >> 172.21.15.117:0/851895810 conn(0x7f4e600fe2f0 msgr2=0x7f4e60106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:13.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.495+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 shutdown_connections 2024-09-11T00:16:13.497 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:16:13.495+0000 7f4e68698640 1 -- 172.21.15.117:0/851895810 wait complete. 2024-09-11T00:16:13.711 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.711 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.711 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:13.712 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:13.712 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:13.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:13.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:13.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:13.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:13.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:14.132 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T00:16:14.134 INFO:tasks.cephadm:Deploying osd.5 on smithi183 with /dev/vg_nvme/lv_3... 2024-09-11T00:16:14.134 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T00:16:14.388 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:16:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:14 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:14 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:14 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:14.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:14 smithi117 ceph-mon[35525]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:14.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:14 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:14.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:14 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:14.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:14 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:14.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:14 smithi183 ceph-mon[35934]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:14.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:14 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:14 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:14 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:14 smithi103 ceph-mon[30578]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:15.909 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:15 smithi183 ceph-mon[35934]: from='osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:16:15.909 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:15 smithi183 ceph-mon[35934]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:16:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:15 smithi117 ceph-mon[35525]: from='osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:16:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:15 smithi117 ceph-mon[35525]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:16:15.938 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:16:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:16:15.627+0000 7fb12c39d640 -1 osd.4 0 log_to_monitors true 2024-09-11T00:16:16.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:15 smithi103 ceph-mon[30578]: from='osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:16:16.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:15 smithi103 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-11T00:16:17.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:16:17.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: from='osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:16:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:16:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:16:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:16:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:16:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:17.083 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:16:17.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-09-11T00:16:17.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:16:17.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: from='osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:16:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:16:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:16:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:16:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:16:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: from='osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T00:16:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:17.363 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:16:17.687 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:16:17.333+0000 7fb128320640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T00:16:17.687 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:16:17.337+0000 7fb12013f640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:16:17.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.866+0000 7f76bd147640 1 -- 172.21.15.183:0/1995990499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 msgr2=0x7f76b8104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:17.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.866+0000 7f76bd147640 1 --2- 172.21.15.183:0/1995990499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8104030 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009960 tx=0x7f76a002a460 comp rx=0 tx=0).stop 2024-09-11T00:16:17.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 -- 172.21.15.183:0/1995990499 shutdown_connections 2024-09-11T00:16:17.875 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 --2- 172.21.15.183:0/1995990499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b8104570 0x7f76b810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 --2- 172.21.15.183:0/1995990499 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8104030 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 --2- 172.21.15.183:0/1995990499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76b8102a50 0x7f76b8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 -- 172.21.15.183:0/1995990499 >> 172.21.15.183:0/1995990499 conn(0x7f76b80fe320 msgr2=0x7f76b8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 -- 172.21.15.183:0/1995990499 shutdown_connections 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 -- 172.21.15.183:0/1995990499 wait complete. 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 Processor -- start 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.867+0000 7f76bd147640 1 -- start start 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b8102a50 0x7f76b8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76b8104570 0x7f76b819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810e800 con 0x7f76b8104570 2024-09-11T00:16:17.876 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810e680 con 0x7f76b8102a50 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76b810e980 con 0x7f76b8103bb0 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:59478/0 (socket says 172.21.15.183:59478) 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 -- 172.21.15.183:0/1682470981 learned_addr learned my addr 172.21.15.183:0/1682470981 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76b6d9d640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b8102a50 0x7f76b8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 -- 172.21.15.183:0/1682470981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b8102a50 msgr2=0x7f76b8198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b8102a50 0x7f76b8198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 -- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76b8104570 msgr2=0x7f76b819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76b8104570 0x7f76b819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 -- 172.21.15.183:0/1682470981 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b819ab10 con 0x7f76b8103bb0 2024-09-11T00:16:17.877 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76b6d9d640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b8102a50 0x7f76b8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76ae59c640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8199630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a0003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76b4d99640 1 -- 172.21.15.183:0/1682470981 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a003f070 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76b4d99640 1 -- 172.21.15.183:0/1682470981 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a0003f30 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b819ada0 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76b4d99640 1 -- 172.21.15.183:0/1682470981 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a0035030 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.868+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b81a1390 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.870+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b81a15f0 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.871+0000 7f76b4d99640 1 -- 172.21.15.183:0/1682470981 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f76a003b4f0 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.871+0000 7f76b4d99640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76a40b52e0 0x7f76a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.871+0000 7f76b4d99640 1 -- 172.21.15.183:0/1682470981 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 3821+0+0 (secure 0 0 0) 0x7f76a01057a0 con 0x7f76b8103bb0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.872+0000 7f76b6d9d640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76a40b52e0 0x7f76a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:17.878 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.872+0000 7f76b6d9d640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76a40b52e0 0x7f76a40b77a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f76a8007eb0 tx=0x7f76a80073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:17.879 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:17.875+0000 7f76b4d99640 1 -- 172.21.15.183:0/1682470981 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76a00cd7f0 con 0x7f76b8103bb0 2024-09-11T00:16:18.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi117 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='osd.4 ' entity='osd.4' 2024-09-11T00:16:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:18.064 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:18.061+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f76b81056a0 con 0x7f76a40b52e0 2024-09-11T00:16:18.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi117 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='osd.4 ' entity='osd.4' 2024-09-11T00:16:18.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi117 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi117 to 8544M 2024-09-11T00:16:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:18.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:18.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:18.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='osd.4 ' entity='osd.4' 2024-09-11T00:16:18.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:19.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:16:19.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] boot 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:16:19.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:18 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:19.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] boot 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:16:19.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:18 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-11T00:16:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: osd.4 [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] boot 2024-09-11T00:16:19.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T00:16:19.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:18 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-11T00:16:20.632 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:20 smithi183 ceph-mon[35934]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:16:20.632 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:20 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/1858863375' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"} : dispatch 2024-09-11T00:16:20.632 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:20 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"} : dispatch 2024-09-11T00:16:20.633 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:20 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"}]': finished 2024-09-11T00:16:20.633 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:20 smithi183 ceph-mon[35934]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:16:20.633 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:20 smithi103 ceph-mon[30578]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:16:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:20 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/1858863375' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"} : dispatch 2024-09-11T00:16:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:20 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"} : dispatch 2024-09-11T00:16:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:20 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"}]': finished 2024-09-11T00:16:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:20 smithi103 ceph-mon[30578]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:16:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:20 smithi117 ceph-mon[35525]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T00:16:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:20 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/1858863375' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"} : dispatch 2024-09-11T00:16:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:20 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"} : dispatch 2024-09-11T00:16:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:20 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d"}]': finished 2024-09-11T00:16:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:20 smithi117 ceph-mon[35525]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T00:16:20.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:21.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:21 smithi103 ceph-mon[30578]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:21.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:21 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/1865616118' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:16:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:21 smithi117 ceph-mon[35525]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:21 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/1865616118' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:16:21.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:21 smithi183 ceph-mon[35934]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:21.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:21 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/1865616118' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:16:23.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:23 smithi103 ceph-mon[30578]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:23.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:23 smithi117 ceph-mon[35525]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:23.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:23 smithi183 ceph-mon[35934]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:25.447 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:25 smithi183 ceph-mon[35934]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:25.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:25 smithi103 ceph-mon[30578]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:25.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:25 smithi117 ceph-mon[35525]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:27.528 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:27 smithi183 ceph-mon[35934]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:27.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:27 smithi103 ceph-mon[30578]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:27 smithi117 ceph-mon[35525]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:29 smithi183 ceph-mon[35934]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:16:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:29.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:29 smithi103 ceph-mon[30578]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:29.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:16:29.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:29 smithi117 ceph-mon[35525]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-11T00:16:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:30.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:30 smithi183 ceph-mon[35934]: Deploying daemon osd.5 on smithi183 2024-09-11T00:16:30.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:30 smithi103 ceph-mon[30578]: Deploying daemon osd.5 on smithi183 2024-09-11T00:16:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:30 smithi117 ceph-mon[35525]: Deploying daemon osd.5 on smithi183 2024-09-11T00:16:31.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:31 smithi183 ceph-mon[35934]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:31.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:31 smithi103 ceph-mon[30578]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:31.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:31 smithi117 ceph-mon[35525]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:32.567 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:32 smithi183 ceph-mon[35934]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:32.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:32 smithi103 ceph-mon[30578]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:32 smithi117 ceph-mon[35525]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:35.623 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:35 smithi183 ceph-mon[35934]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:35.623 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:35.623 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:35.623 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:35.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:35 smithi103 ceph-mon[30578]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:35.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:35.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:35.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:35 smithi117 ceph-mon[35525]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:36.646 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:36 smithi183 ceph-mon[35934]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:36.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:36 smithi103 ceph-mon[30578]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:36 smithi117 ceph-mon[35525]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:39.101 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.101 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.102 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:39.102 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:39.102 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.102 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:38 smithi183 ceph-mon[35934]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:39.143 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 5 on host 'smithi183' 2024-09-11T00:16:39.143 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.141+0000 7f76b4d99640 1 -- 172.21.15.183:0/1682470981 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76b81056a0 con 0x7f76a40b52e0 2024-09-11T00:16:39.145 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76a40b52e0 msgr2=0x7f76a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.145 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76a40b52e0 0x7f76a40b77a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f76a8007eb0 tx=0x7f76a80073c0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.145 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 msgr2=0x7f76b8199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:39.145 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8199630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f76a0000c00 tx=0x7f76a0003c80 comp rx=0 tx=0).stop 2024-09-11T00:16:39.145 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 shutdown_connections 2024-09-11T00:16:39.145 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76a40b52e0 0x7f76a40b77a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.145 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76b8104570 0x7f76b819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76b8103bb0 0x7f76b8199630 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 --2- 172.21.15.183:0/1682470981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76b8102a50 0x7f76b8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:39.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.143+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 >> 172.21.15.183:0/1682470981 conn(0x7f76b80fe320 msgr2=0x7f76b8104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:39.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.144+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 shutdown_connections 2024-09-11T00:16:39.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:16:39.144+0000 7f76bd147640 1 -- 172.21.15.183:0/1682470981 wait complete. 2024-09-11T00:16:39.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:39.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:39.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:38 smithi103 ceph-mon[30578]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:39.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:39.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:39.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:39.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:38 smithi117 ceph-mon[35525]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:39.833 DEBUG:teuthology.orchestra.run.smithi183:osd.5> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T00:16:39.836 INFO:tasks.cephadm:Deploying osd.6 on smithi103 with /dev/vg_nvme/lv_2... 2024-09-11T00:16:39.836 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:16:40.108 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:40.108 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:40.108 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:40.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:40.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:40.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:40.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:16:40.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:40.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:40.213 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:16:41.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:40 smithi103 ceph-mon[30578]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:41.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:40 smithi117 ceph-mon[35525]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:41.234 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:40 smithi183 ceph-mon[35934]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:41.531 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:16:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:16:41.479+0000 7f01b6a5c640 -1 osd.5 0 log_to_monitors true 2024-09-11T00:16:41.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:41 smithi103 ceph-mon[30578]: from='osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:16:41.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:41 smithi103 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:16:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:41 smithi117 ceph-mon[35525]: from='osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:16:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:41 smithi117 ceph-mon[35525]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:16:42.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:41 smithi183 ceph-mon[35934]: from='osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:16:42.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:41 smithi183 ceph-mon[35934]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:43.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:43.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:43.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:43.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:16:43.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:16:43.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:43.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:43.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.280 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:16:42.912+0000 7f01b29df640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T00:16:43.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:16:42.917+0000 7f01ae806640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:43.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:16:43.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:43.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:16:43.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:16:43.610 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:16:43.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-09-11T00:16:44.003 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:16:44.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi183 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] boot 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:16:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi183 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] boot 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:16:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi183 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: osd.5 [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] boot 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T00:16:44.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-11T00:16:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 -- 172.21.15.103:0/4196316278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 msgr2=0x7f713810ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 --2- 172.21.15.103:0/4196316278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 0x7f713810ad30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7130009cd0 tx=0x7f713002a8e0 comp rx=0 tx=0).stop 2024-09-11T00:16:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 -- 172.21.15.103:0/4196316278 shutdown_connections 2024-09-11T00:16:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 --2- 172.21.15.103:0/4196316278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 0x7f713810ad30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:44.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 --2- 172.21.15.103:0/4196316278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71381039f0 0x7f7138103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 --2- 172.21.15.103:0/4196316278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102890 0x7f7138102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 -- 172.21.15.103:0/4196316278 >> 172.21.15.103:0/4196316278 conn(0x7f71380fe180 msgr2=0x7f71381005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 -- 172.21.15.103:0/4196316278 shutdown_connections 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.617+0000 7f713de00640 1 -- 172.21.15.103:0/4196316278 wait complete. 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.618+0000 7f713de00640 1 Processor -- start 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.618+0000 7f713de00640 1 -- start start 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102890 0x7f7138198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71381039f0 0x7f71381991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 0x7f7138199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713de00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f713810e4f0 con 0x7f71381043b0 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713de00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f713810e370 con 0x7f71381039f0 2024-09-11T00:16:44.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713de00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f713810e670 con 0x7f7138102890 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102890 0x7f7138198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713d400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 0x7f7138199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102890 0x7f7138198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39496/0 (socket says 172.21.15.103:39496) 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713ca00640 1 -- 172.21.15.103:0/2058994613 learned_addr learned my addr 172.21.15.103:0/2058994613 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f712fe00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71381039f0 0x7f71381991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713d400640 1 -- 172.21.15.103:0/2058994613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102890 msgr2=0x7f7138198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713d400640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102890 0x7f7138198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713d400640 1 -- 172.21.15.103:0/2058994613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71381039f0 msgr2=0x7f71381991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713d400640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71381039f0 0x7f71381991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f713d400640 1 -- 172.21.15.103:0/2058994613 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713819a6d0 con 0x7f71381043b0 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.619+0000 7f712fe00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71381039f0 0x7f71381991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:16:44.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.620+0000 7f713d400640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 0x7f7138199e30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7130006010 tx=0x7f7130003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.620+0000 7f7136000640 1 -- 172.21.15.103:0/2058994613 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130004240 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.620+0000 7f7136000640 1 -- 172.21.15.103:0/2058994613 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71300305a0 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.620+0000 7f7136000640 1 -- 172.21.15.103:0/2058994613 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130045670 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.620+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713819a960 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.620+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71381a0ed0 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.621+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7138109b40 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.623+0000 7f7136000640 1 -- 172.21.15.103:0/2058994613 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f713004d070 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.625+0000 7f7136000640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f711c0b5490 0x7f711c0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.625+0000 7f7136000640 1 -- 172.21.15.103:0/2058994613 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4533+0+0 (secure 0 0 0) 0x7f713002e030 con 0x7f71381043b0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.625+0000 7f713ca00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f711c0b5490 0x7f711c0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.626+0000 7f713ca00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f711c0b5490 0x7f711c0b7950 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7128005fd0 tx=0x7f712800b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:16:44.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.629+0000 7f7136000640 1 -- 172.21.15.103:0/2058994613 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f713004a060 con 0x7f71381043b0 2024-09-11T00:16:44.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:16:44.823+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7138063590 con 0x7f711c0b5490 2024-09-11T00:16:45.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:45 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:16:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:45 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:16:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:45 smithi103 ceph-mon[30578]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:16:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:45 smithi103 ceph-mon[30578]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:16:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:16:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:45.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:45 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:16:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:45 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:16:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:45 smithi183 ceph-mon[35934]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:16:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:45 smithi183 ceph-mon[35934]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:16:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:16:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:45 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:16:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:45 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:16:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:45 smithi117 ceph-mon[35525]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T00:16:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:45 smithi117 ceph-mon[35525]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:16:45.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:16:45.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:46.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:46 smithi183 ceph-mon[35934]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:16:46.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:46 smithi183 ceph-mon[35934]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:16:46.284 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:46 smithi103 ceph-mon[30578]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:16:46.284 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:46 smithi103 ceph-mon[30578]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:16:46.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:46 smithi117 ceph-mon[35525]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:16:46.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:46 smithi117 ceph-mon[35525]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T00:16:47.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:47 smithi183 ceph-mon[35934]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:47.300 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:47 smithi103 ceph-mon[30578]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:47 smithi117 ceph-mon[35525]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:48.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:48 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2303061516' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe"} : dispatch 2024-09-11T00:16:48.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:48 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2303061516' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe"}]': finished 2024-09-11T00:16:48.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:48 smithi117 ceph-mon[35525]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:16:48.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:48 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:16:48.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:48 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2303061516' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe"} : dispatch 2024-09-11T00:16:48.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:48 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2303061516' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe"}]': finished 2024-09-11T00:16:48.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:48 smithi183 ceph-mon[35934]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:16:48.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:48 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:16:48.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:48 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2303061516' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe"} : dispatch 2024-09-11T00:16:48.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:48 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2303061516' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe"}]': finished 2024-09-11T00:16:48.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:48 smithi103 ceph-mon[30578]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T00:16:48.557 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:48 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:16:49.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:49 smithi103 ceph-mon[30578]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-11T00:16:49.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:49 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3045521733' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:16:49.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:49 smithi117 ceph-mon[35525]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-11T00:16:49.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:49 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3045521733' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:16:49.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:49 smithi183 ceph-mon[35934]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-11T00:16:49.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:49 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3045521733' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:16:50.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:50 smithi103 ceph-mon[30578]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:16:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:50 smithi117 ceph-mon[35525]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:16:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:50 smithi183 ceph-mon[35934]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:16:52.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:52 smithi103 ceph-mon[30578]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:16:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:52 smithi117 ceph-mon[35525]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:16:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:52 smithi183 ceph-mon[35934]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:16:54.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:54 smithi103 ceph-mon[30578]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:16:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:54 smithi117 ceph-mon[35525]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:16:54.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:54 smithi183 ceph-mon[35934]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:16:57.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:57 smithi103 ceph-mon[30578]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:16:57.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:57 smithi117 ceph-mon[35525]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:16:57.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:57 smithi183 ceph-mon[35934]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:16:58.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:16:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:58 smithi103 ceph-mon[30578]: Deploying daemon osd.6 on smithi103 2024-09-11T00:16:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:16:58 smithi103 ceph-mon[30578]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:16:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:58 smithi117 ceph-mon[35525]: Deploying daemon osd.6 on smithi103 2024-09-11T00:16:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:16:58 smithi117 ceph-mon[35525]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:16:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-11T00:16:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:16:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:58 smithi183 ceph-mon[35934]: Deploying daemon osd.6 on smithi103 2024-09-11T00:16:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:16:58 smithi183 ceph-mon[35934]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:00.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:00 smithi103 ceph-mon[30578]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:00.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:00 smithi117 ceph-mon[35525]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:00 smithi183 ceph-mon[35934]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:02.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:02 smithi103 ceph-mon[30578]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:02 smithi117 ceph-mon[35525]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:02 smithi183 ceph-mon[35934]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:04.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:04 smithi103 ceph-mon[30578]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:04 smithi117 ceph-mon[35525]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:04 smithi183 ceph-mon[35934]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:06.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:06.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:06.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:06.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:06.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:06.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:07.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:07 smithi103 ceph-mon[30578]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:07.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:07 smithi117 ceph-mon[35525]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:07.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:07 smithi183 ceph-mon[35934]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:08 smithi103 ceph-mon[30578]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:08.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:08 smithi117 ceph-mon[35525]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:08 smithi183 ceph-mon[35934]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:10.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:10.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:10.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:10 smithi103 ceph-mon[30578]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:10.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:10.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:10.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:10 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:10 smithi117 ceph-mon[35525]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:10 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:11.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:11.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:11.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:10 smithi183 ceph-mon[35934]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:11.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:11.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:11.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:10 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:11.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.104+0000 7f7136000640 1 -- 172.21.15.103:0/2058994613 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7138063590 con 0x7f711c0b5490 2024-09-11T00:17:11.107 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 6 on host 'smithi103' 2024-09-11T00:17:11.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f711c0b5490 msgr2=0x7f711c0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f711c0b5490 0x7f711c0b7950 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7128005fd0 tx=0x7f712800b480 comp rx=0 tx=0).stop 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 msgr2=0x7f7138199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 0x7f7138199e30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7130006010 tx=0x7f7130003b80 comp rx=0 tx=0).stop 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 shutdown_connections 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f711c0b5490 0x7f711c0b7950 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71381043b0 0x7f7138199e30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71381039f0 0x7f71381991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 --2- 172.21.15.103:0/2058994613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7138102890 0x7f7138198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.106+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 >> 172.21.15.103:0/2058994613 conn(0x7f71380fe180 msgr2=0x7f7138106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.107+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 shutdown_connections 2024-09-11T00:17:11.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:17:11.107+0000 7f713de00640 1 -- 172.21.15.103:0/2058994613 wait complete. 2024-09-11T00:17:11.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:11.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:11.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:11 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:11.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:11 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:11.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:11 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:11.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:11 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:12.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:12.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:12.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:11 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:12.188 DEBUG:teuthology.orchestra.run.smithi103:osd.6> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T00:17:12.193 INFO:tasks.cephadm:Deploying osd.7 on smithi117 with /dev/vg_nvme/lv_2... 2024-09-11T00:17:12.193 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:17:12.456 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:17:12.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:12 smithi103 ceph-mon[30578]: from='osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:17:12.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:12 smithi103 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:17:12.827 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:12 smithi103 ceph-mon[30578]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:12.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:12 smithi117 ceph-mon[35525]: from='osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:17:12.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:12 smithi117 ceph-mon[35525]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:17:12.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:12 smithi117 ceph-mon[35525]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:13.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:12 smithi183 ceph-mon[35934]: from='osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:17:13.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:12 smithi183 ceph-mon[35934]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-11T00:17:13.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:12 smithi183 ceph-mon[35934]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:13.915 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:13 smithi103 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:17:13.915 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:13 smithi103 ceph-mon[30578]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:17:13.915 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:13 smithi103 ceph-mon[30578]: from='osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:17:13.916 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:13 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:13.916 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:13 smithi103 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:17:13.916 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:17:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:17:13.679+0000 7fb236200640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T00:17:13.916 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:17:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:17:13.690+0000 7fb231200640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:17:13.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:13 smithi117 ceph-mon[35525]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:17:13.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:13 smithi117 ceph-mon[35525]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:17:13.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:13 smithi117 ceph-mon[35525]: from='osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:17:13.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:13 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:13.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:13 smithi117 ceph-mon[35525]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:17:14.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:13 smithi183 ceph-mon[35934]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T00:17:14.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:13 smithi183 ceph-mon[35934]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T00:17:14.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:13 smithi183 ceph-mon[35934]: from='osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:17:14.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:13 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:14.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:13 smithi183 ceph-mon[35934]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:17:14.761 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:14 smithi117 ceph-mon[35525]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:17:14.761 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:14 smithi117 ceph-mon[35525]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:17:14.761 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:14 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:14.761 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:14 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:14.761 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:14 smithi117 ceph-mon[35525]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:14 smithi103 ceph-mon[30578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:17:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:14 smithi103 ceph-mon[30578]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:17:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:14 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:14 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:14 smithi103 ceph-mon[30578]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:15.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:14 smithi183 ceph-mon[35934]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:17:15.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:14 smithi183 ceph-mon[35934]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T00:17:15.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:14 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:15.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:14 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:15.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:14 smithi183 ceph-mon[35934]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-11T00:17:15.056 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:17:15.084 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-09-11T00:17:15.327 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:17:15.723 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:15 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:17:15.724 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:15 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:17:15.724 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:15 smithi117 ceph-mon[35525]: osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] boot 2024-09-11T00:17:15.724 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:15 smithi117 ceph-mon[35525]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:17:15.724 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:15.832 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 -- 172.21.15.117:0/419963360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210102a20 msgr2=0x7fa210102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 --2- 172.21.15.117:0/419963360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210102a20 0x7fa210102e20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa20c009930 tx=0x7fa20c02a420 comp rx=0 tx=0).stop 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 -- 172.21.15.117:0/419963360 shutdown_connections 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 --2- 172.21.15.117:0/419963360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa210104540 0x7fa21010aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 --2- 172.21.15.117:0/419963360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa210103b80 0x7fa210104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 --2- 172.21.15.117:0/419963360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210102a20 0x7fa210102e20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 -- 172.21.15.117:0/419963360 >> 172.21.15.117:0/419963360 conn(0x7fa2100fe2f0 msgr2=0x7fa210100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 -- 172.21.15.117:0/419963360 shutdown_connections 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.804+0000 7fa21856d640 1 -- 172.21.15.117:0/419963360 wait complete. 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 Processor -- start 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 -- start start 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa210102a20 0x7fa210198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210103b80 0x7fa210199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:15.833 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa210104540 0x7fa21019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa21010e680 con 0x7fa210102a20 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa21010e500 con 0x7fa210103b80 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa21856d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa21010e800 con 0x7fa210104540 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210103b80 0x7fa210199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210103b80 0x7fa210199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:39536/0 (socket says 172.21.15.117:39536) 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 -- 172.21.15.117:0/1986269618 learned_addr learned my addr 172.21.15.117:0/1986269618 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa216b0a640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa210104540 0x7fa21019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa216309640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa210102a20 0x7fa210198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 -- 172.21.15.117:0/1986269618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa210104540 msgr2=0x7fa21019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa210104540 0x7fa21019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 -- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa210102a20 msgr2=0x7fa210198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:15.834 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa210102a20 0x7fa210198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa215b08640 1 -- 172.21.15.117:0/1986269618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21019a940 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.805+0000 7fa216b0a640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa210104540 0x7fa21019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.806+0000 7fa216309640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa210102a20 0x7fa210198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.806+0000 7fa215b08640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210103b80 0x7fa210199460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa20000d950 tx=0x7fa20000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.806+0000 7fa2077fe640 1 -- 172.21.15.117:0/1986269618 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200017030 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.806+0000 7fa2077fe640 1 -- 172.21.15.117:0/1986269618 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa200004020 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.806+0000 7fa2077fe640 1 -- 172.21.15.117:0/1986269618 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2000047a0 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.806+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21019ac30 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.806+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2101a1310 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.807+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa210109d40 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.808+0000 7fa2077fe640 1 -- 172.21.15.117:0/1986269618 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fa200009a30 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.809+0000 7fa2077fe640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1fc0b52e0 0x7fa1fc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.809+0000 7fa2077fe640 1 -- 172.21.15.117:0/1986269618 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7fa2000e5080 con 0x7fa210103b80 2024-09-11T00:17:15.835 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.810+0000 7fa216309640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1fc0b52e0 0x7fa1fc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:15.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.810+0000 7fa216309640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1fc0b52e0 0x7fa1fc0b77a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000c00 tx=0x7fa20c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:15.836 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.812+0000 7fa2077fe640 1 -- 172.21.15.117:0/1986269618 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa2000acc50 con 0x7fa210103b80 2024-09-11T00:17:16.000 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:15.998+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa210063590 con 0x7fa1fc0b52e0 2024-09-11T00:17:16.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:15 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:17:16.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:15 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:17:16.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:15 smithi183 ceph-mon[35934]: osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] boot 2024-09-11T00:17:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:15 smithi183 ceph-mon[35934]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:17:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:16.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:15 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:17:16.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:15 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:17:16.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:15 smithi103 ceph-mon[30578]: osd.6 [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] boot 2024-09-11T00:17:16.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:15 smithi103 ceph-mon[30578]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T00:17:16.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-11T00:17:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:17:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:17:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:17:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:17:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi103 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:16.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:16 smithi117 ceph-mon[35525]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi103 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:17.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:17.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:16 smithi183 ceph-mon[35934]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi103 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi103 to 5696M 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:17.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:16 smithi103 ceph-mon[30578]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:18.671 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:18 smithi117 ceph-mon[35525]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:18.671 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:18 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/1763825811' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"} : dispatch 2024-09-11T00:17:18.671 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:18 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"} : dispatch 2024-09-11T00:17:18.671 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:18 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"}]': finished 2024-09-11T00:17:18.671 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:18 smithi117 ceph-mon[35525]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:17:18.671 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:18 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:18.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:18 smithi183 ceph-mon[35934]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:18 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/1763825811' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"} : dispatch 2024-09-11T00:17:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:18 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"} : dispatch 2024-09-11T00:17:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:18 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"}]': finished 2024-09-11T00:17:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:18 smithi183 ceph-mon[35934]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:17:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:18 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:18.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:18 smithi103 ceph-mon[30578]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:18.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:18 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/1763825811' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"} : dispatch 2024-09-11T00:17:18.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:18 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"} : dispatch 2024-09-11T00:17:18.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:18 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "504cd71d-934e-4a6e-9ed2-5fc4a758800e"}]': finished 2024-09-11T00:17:18.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:18 smithi103 ceph-mon[30578]: osdmap e44: 8 total, 7 up, 8 in 2024-09-11T00:17:18.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:18 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:19.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:19 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/1656546951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:17:19.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:19 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/1656546951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:17:19.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:19 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/1656546951' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:17:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:20 smithi117 ceph-mon[35525]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:20 smithi183 ceph-mon[35934]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:20.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:20 smithi103 ceph-mon[30578]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:22 smithi117 ceph-mon[35525]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:22 smithi183 ceph-mon[35934]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:22.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:22 smithi103 ceph-mon[30578]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:24.502 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:24 smithi117 ceph-mon[35525]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:24.731 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:24 smithi103 ceph-mon[30578]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:24 smithi183 ceph-mon[35934]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:26.686 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:26 smithi117 ceph-mon[35525]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:26 smithi183 ceph-mon[35934]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:26.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:26 smithi103 ceph-mon[30578]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:27.682 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:17:27.682 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:27.682 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:27 smithi117 ceph-mon[35525]: Deploying daemon osd.7 on smithi117 2024-09-11T00:17:27.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:17:27.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:27.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:27 smithi183 ceph-mon[35934]: Deploying daemon osd.7 on smithi117 2024-09-11T00:17:27.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-11T00:17:27.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:27.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:27 smithi103 ceph-mon[30578]: Deploying daemon osd.7 on smithi117 2024-09-11T00:17:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:28 smithi117 ceph-mon[35525]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:28 smithi183 ceph-mon[35934]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:28 smithi103 ceph-mon[30578]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:30 smithi117 ceph-mon[35525]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:30 smithi183 ceph-mon[35934]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:30.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:30 smithi103 ceph-mon[30578]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:32.544 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:32 smithi117 ceph-mon[35525]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:32 smithi183 ceph-mon[35934]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:32.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:32 smithi103 ceph-mon[30578]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:33 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:33 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:33 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:33.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:33 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:33.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:33 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:33.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:33 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:33.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:33 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:33.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:33 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:33.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:33 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:34 smithi117 ceph-mon[35525]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:34 smithi183 ceph-mon[35934]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:34.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:34 smithi103 ceph-mon[30578]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:37.022 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.020+0000 7fa2077fe640 1 -- 172.21.15.117:0/1986269618 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa210063590 con 0x7fa1fc0b52e0 2024-09-11T00:17:37.023 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 7 on host 'smithi117' 2024-09-11T00:17:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.022+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1fc0b52e0 msgr2=0x7fa1fc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:37.024 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.022+0000 7fa21856d640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1fc0b52e0 0x7fa1fc0b77a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa20c000c00 tx=0x7fa20c009560 comp rx=0 tx=0).stop 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.022+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210103b80 msgr2=0x7fa210199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.022+0000 7fa21856d640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210103b80 0x7fa210199460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa20000d950 tx=0x7fa20000de10 comp rx=0 tx=0).stop 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 shutdown_connections 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1fc0b52e0 0x7fa1fc0b77a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa210104540 0x7fa21019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa210103b80 0x7fa210199460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 --2- 172.21.15.117:0/1986269618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa210102a20 0x7fa210198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 >> 172.21.15.117:0/1986269618 conn(0x7fa2100fe2f0 msgr2=0x7fa210106f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 shutdown_connections 2024-09-11T00:17:37.025 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:17:37.023+0000 7fa21856d640 1 -- 172.21.15.117:0/1986269618 wait complete. 2024-09-11T00:17:37.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:37.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:37.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:36 smithi117 ceph-mon[35525]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:37.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:37.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:37.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:36 smithi183 ceph-mon[35934]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:37.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:37.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:37.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:37.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:36 smithi103 ceph-mon[30578]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:37.680 DEBUG:teuthology.orchestra.run.smithi117:osd.7> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T00:17:37.682 INFO:tasks.cephadm:Deploying osd.8 on smithi183 with /dev/vg_nvme/lv_2... 2024-09-11T00:17:37.682 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T00:17:37.938 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:17:38.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:37 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:38.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:37 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:38.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:37 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:38.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:37 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:38.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:37 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:38.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:37 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:38.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:37 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:38.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:37 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:38.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:37 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:39.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:38 smithi117 ceph-mon[35525]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:39.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:38 smithi183 ceph-mon[35934]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:39.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:38 smithi103 ceph-mon[30578]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:39.687 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:17:39 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:17:39.461+0000 7ff38aa84640 -1 osd.7 0 log_to_monitors true 2024-09-11T00:17:40.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:39 smithi183 ceph-mon[35934]: from='osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:17:40.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:39 smithi183 ceph-mon[35934]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:17:40.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:39 smithi117 ceph-mon[35525]: from='osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:17:40.262 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:39 smithi117 ceph-mon[35525]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:17:40.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:39 smithi103 ceph-mon[30578]: from='osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:17:40.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:39 smithi103 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-11T00:17:40.557 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:17:40.589 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-09-11T00:17:40.856 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:17:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:17:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:17:41.232 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:41.233 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.321 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:17:41.321 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:41.322 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:41.323 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- 172.21.15.183:0/3682062981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7860108730 msgr2=0x7f786010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 --2- 172.21.15.183:0/3682062981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7860108730 0x7f786010abf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7850009900 tx=0x7f785002a410 comp rx=0 tx=0).stop 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- 172.21.15.183:0/3682062981 shutdown_connections 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 --2- 172.21.15.183:0/3682062981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7860108730 0x7f786010abf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 --2- 172.21.15.183:0/3682062981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7860105b80 0x7f7860107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 --2- 172.21.15.183:0/3682062981 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f786006b6e0 0x7f78601055f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- 172.21.15.183:0/3682062981 >> 172.21.15.183:0/3682062981 conn(0x7f78600fd1e0 msgr2=0x7f78600ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- 172.21.15.183:0/3682062981 shutdown_connections 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- 172.21.15.183:0/3682062981 wait complete. 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 Processor -- start 2024-09-11T00:17:41.340 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- start start 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f786006b6e0 0x7f7860198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7860105b80 0x7f7860199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7860108730 0x7f786019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f786010e700 con 0x7f7860108730 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f786010e580 con 0x7f7860105b80 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.317+0000 7f786700c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f786010e880 con 0x7f786006b6e0 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f7864da8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f786006b6e0 0x7f7860198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f7864da8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f786006b6e0 0x7f7860198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:36770/0 (socket says 172.21.15.183:36770) 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f7864da8640 1 -- 172.21.15.183:0/2227615184 learned_addr learned my addr 172.21.15.183:0/2227615184 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785ffff640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7860105b80 0x7f7860199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:41.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f78655a9640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7860108730 0x7f786019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785ffff640 1 -- 172.21.15.183:0/2227615184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f786006b6e0 msgr2=0x7f7860198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785ffff640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f786006b6e0 0x7f7860198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785ffff640 1 -- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7860108730 msgr2=0x7f786019a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785ffff640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7860108730 0x7f786019a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785ffff640 1 -- 172.21.15.183:0/2227615184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786019a900 con 0x7f7860105b80 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785ffff640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7860105b80 0x7f7860199420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7848005fd0 tx=0x7f784800df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785dffb640 1 -- 172.21.15.183:0/2227615184 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848017030 con 0x7f7860105b80 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786019abf0 con 0x7f7860105b80 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785dffb640 1 -- 172.21.15.183:0/2227615184 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7848003e20 con 0x7f7860105b80 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f785dffb640 1 -- 172.21.15.183:0/2227615184 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848010460 con 0x7f7860105b80 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.318+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78601a12d0 con 0x7f7860105b80 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.320+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78600fecd0 con 0x7f7860105b80 2024-09-11T00:17:41.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.321+0000 7f785dffb640 1 -- 172.21.15.183:0/2227615184 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f7848024060 con 0x7f7860105b80 2024-09-11T00:17:41.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.322+0000 7f785dffb640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f784c0b52e0 0x7f784c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:17:41.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.322+0000 7f785dffb640 1 -- 172.21.15.183:0/2227615184 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5117+0+0 (secure 0 0 0) 0x7f78480e4e00 con 0x7f7860105b80 2024-09-11T00:17:41.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.322+0000 7f7864da8640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f784c0b52e0 0x7f784c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:17:41.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.323+0000 7f7864da8640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f784c0b52e0 0x7f784c0b77a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7858009a40 tx=0x7f78580091c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:17:41.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.326+0000 7f785dffb640 1 -- 172.21.15.183:0/2227615184 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f78480ac8a0 con 0x7f7860105b80 2024-09-11T00:17:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T00:17:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:17:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:17:41.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:41.523 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:17:41.522+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7860063590 con 0x7f784c0b52e0 2024-09-11T00:17:41.719 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:17:41.626+0000 7ff38721b640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T00:17:41.719 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:17:41.641+0000 7ff37e826640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:17:42.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi117 2024-09-11T00:17:42.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-11T00:17:42.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:17:42.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:17:42.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:42.185 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:42.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:17:42.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:17:42.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:42.186 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:41 smithi183 ceph-mon[35934]: from='osd.7 ' entity='osd.7' 2024-09-11T00:17:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi117 2024-09-11T00:17:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-11T00:17:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:17:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:17:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:17:42.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:17:42.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:42.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:41 smithi117 ceph-mon[35525]: from='osd.7 ' entity='osd.7' 2024-09-11T00:17:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi117 2024-09-11T00:17:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi117 to 5696M 2024-09-11T00:17:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:17:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T00:17:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:42.346 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:42.346 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:17:42.346 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:17:42.346 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:42.346 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:41 smithi103 ceph-mon[30578]: from='osd.7 ' entity='osd.7' 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: from='client.24452 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] boot 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:43.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:42 smithi183 ceph-mon[35934]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:43.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:17:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:17:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: from='client.24452 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:17:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] boot 2024-09-11T00:17:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:17:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:42 smithi103 ceph-mon[30578]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: from='client.24452 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: osd.7 [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] boot 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: osdmap e47: 8 total, 8 up, 8 in 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-11T00:17:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:42 smithi117 ceph-mon[35525]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-11T00:17:44.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:43 smithi103 ceph-mon[30578]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:17:44.261 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:43 smithi183 ceph-mon[35934]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:17:44.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:43 smithi117 ceph-mon[35525]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T00:17:45.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T00:17:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/3635462706' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"} : dispatch 2024-09-11T00:17:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"} : dispatch 2024-09-11T00:17:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"}]': finished 2024-09-11T00:17:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:17:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:17:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:17:45.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:44 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/4070708742' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:17:45.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T00:17:45.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/3635462706' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"} : dispatch 2024-09-11T00:17:45.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"} : dispatch 2024-09-11T00:17:45.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"}]': finished 2024-09-11T00:17:45.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:17:45.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:17:45.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:17:45.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:44 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/4070708742' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/3635462706' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"} : dispatch 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"} : dispatch 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ede29a2-c541-4834-b320-cd6f66b7846a"}]': finished 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: osdmap e50: 9 total, 8 up, 9 in 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:17:45.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:44 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/4070708742' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:17:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:46 smithi117 ceph-mon[35525]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:17:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:46 smithi183 ceph-mon[35934]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:17:46.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:46 smithi103 ceph-mon[30578]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:17:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:48 smithi117 ceph-mon[35525]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:17:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:48 smithi183 ceph-mon[35934]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:17:48.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:48 smithi103 ceph-mon[30578]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T00:17:50.584 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:50 smithi183 ceph-mon[35934]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T00:17:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:50 smithi117 ceph-mon[35525]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T00:17:50.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:50 smithi103 ceph-mon[30578]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-11T00:17:52.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:52 smithi117 ceph-mon[35525]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T00:17:52.689 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:52 smithi183 ceph-mon[35934]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T00:17:52.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:52 smithi103 ceph-mon[30578]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-11T00:17:53.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:17:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:53.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:53 smithi103 ceph-mon[30578]: Deploying daemon osd.8 on smithi183 2024-09-11T00:17:53.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:17:53.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:53.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:53 smithi117 ceph-mon[35525]: Deploying daemon osd.8 on smithi183 2024-09-11T00:17:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-11T00:17:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:17:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:53 smithi183 ceph-mon[35934]: Deploying daemon osd.8 on smithi183 2024-09-11T00:17:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:54 smithi117 ceph-mon[35525]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T00:17:54.698 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:54 smithi183 ceph-mon[35934]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T00:17:54.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:54 smithi103 ceph-mon[30578]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-11T00:17:56.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:56 smithi117 ceph-mon[35525]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:17:56.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:56 smithi183 ceph-mon[35934]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:17:56.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:56 smithi103 ceph-mon[30578]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:17:58.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:58 smithi183 ceph-mon[35934]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:17:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:58 smithi117 ceph-mon[35525]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:17:58.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:58 smithi103 ceph-mon[30578]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:17:59.519 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:59 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:59.519 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:59 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:59.519 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:17:59 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:59 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:59 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:17:59 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:59.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:59 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:17:59.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:59 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:17:59.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:17:59 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:00.541 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:00 smithi183 ceph-mon[35934]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:00.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:00 smithi103 ceph-mon[30578]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:00 smithi117 ceph-mon[35525]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:02.570 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.568+0000 7f785dffb640 1 -- 172.21.15.183:0/2227615184 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7860063590 con 0x7f784c0b52e0 2024-09-11T00:18:02.570 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 8 on host 'smithi183' 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f784c0b52e0 msgr2=0x7f784c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f784c0b52e0 0x7f784c0b77a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7858009a40 tx=0x7f78580091c0 comp rx=0 tx=0).stop 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7860105b80 msgr2=0x7f7860199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7860105b80 0x7f7860199420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7848005fd0 tx=0x7f784800df40 comp rx=0 tx=0).stop 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 shutdown_connections 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f784c0b52e0 0x7f784c0b77a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7860108730 0x7f786019a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7860105b80 0x7f7860199420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 --2- 172.21.15.183:0/2227615184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f786006b6e0 0x7f7860198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:02.572 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 >> 172.21.15.183:0/2227615184 conn(0x7f78600fd1e0 msgr2=0x7f7860103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:02.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 shutdown_connections 2024-09-11T00:18:02.573 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:18:02.570+0000 7f786700c640 1 -- 172.21.15.183:0/2227615184 wait complete. 2024-09-11T00:18:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:03.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:03.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:03.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:03.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:03.216 DEBUG:teuthology.orchestra.run.smithi183:osd.8> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T00:18:03.218 INFO:tasks.cephadm:Deploying osd.9 on smithi103 with /dev/vg_nvme/lv_1... 2024-09-11T00:18:03.218 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:18:03.602 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:04 smithi117 ceph-mon[35525]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:04.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:04 smithi183 ceph-mon[35934]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:04.761 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:18:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:18:04.621+0000 7fad256e3640 -1 osd.8 0 log_to_monitors true 2024-09-11T00:18:04.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:04 smithi103 ceph-mon[30578]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:05.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:05 smithi103 ceph-mon[30578]: from='osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:18:05.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:05 smithi103 ceph-mon[30578]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:18:05.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:05 smithi117 ceph-mon[35525]: from='osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:18:05.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:05 smithi117 ceph-mon[35525]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:18:05.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:05 smithi183 ceph-mon[35934]: from='osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:18:05.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:05 smithi183 ceph-mon[35934]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-11T00:18:06.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi183 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-11T00:18:06.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:06.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:06.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:06 smithi103 ceph-mon[30578]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:06.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi183 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-11T00:18:06.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:06.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:06.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:06 smithi183 ceph-mon[35934]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:06.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T00:18:06.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T00:18:06.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi183 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:06.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:06.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:06 smithi117 ceph-mon[35525]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:07.143 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:18:07.178 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-09-11T00:18:07.451 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:18:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:18:07.082+0000 7fad21e7a640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T00:18:07.451 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:18:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:18:07.093+0000 7fad19485640 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:18:07.566 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:18:07.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:07 smithi103 ceph-mon[30578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:18:07.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:07 smithi103 ceph-mon[30578]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T00:18:07.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:07 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:07.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:07 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:07.705 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:07 smithi103 ceph-mon[30578]: from='osd.8 ' entity='osd.8' 2024-09-11T00:18:07.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:07 smithi183 ceph-mon[35934]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:18:07.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:07 smithi183 ceph-mon[35934]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T00:18:07.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:07 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:07.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:07 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:07.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:07 smithi183 ceph-mon[35934]: from='osd.8 ' entity='osd.8' 2024-09-11T00:18:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:07 smithi117 ceph-mon[35525]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:18:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:07 smithi117 ceph-mon[35525]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T00:18:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:07 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:07 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:07 smithi117 ceph-mon[35525]: from='osd.8 ' entity='osd.8' 2024-09-11T00:18:08.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 -- 172.21.15.103:0/4236526848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060106780 msgr2=0x7ff06010d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:08.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 --2- 172.21.15.103:0/4236526848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060106780 0x7ff06010d100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff058009cd0 tx=0x7ff05802a8e0 comp rx=0 tx=0).stop 2024-09-11T00:18:08.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 -- 172.21.15.103:0/4236526848 shutdown_connections 2024-09-11T00:18:08.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 --2- 172.21.15.103:0/4236526848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060106780 0x7ff06010d100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:08.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 --2- 172.21.15.103:0/4236526848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff060105dc0 0x7ff060106240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:08.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 --2- 172.21.15.103:0/4236526848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060104c60 0x7ff060105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:08.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 -- 172.21.15.103:0/4236526848 >> 172.21.15.103:0/4236526848 conn(0x7ff060100510 msgr2=0x7ff060102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:08.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 -- 172.21.15.103:0/4236526848 shutdown_connections 2024-09-11T00:18:08.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.209+0000 7ff065c00640 1 -- 172.21.15.103:0/4236526848 wait complete. 2024-09-11T00:18:08.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.210+0000 7ff065c00640 1 Processor -- start 2024-09-11T00:18:08.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.210+0000 7ff065c00640 1 -- start start 2024-09-11T00:18:08.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff065c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060104c60 0x7ff06019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:08.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff065c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105dc0 0x7ff06019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:08.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff065c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff060106780 0x7ff06019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:08.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff065c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0601108e0 con 0x7ff060104c60 2024-09-11T00:18:08.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff065c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff060110760 con 0x7ff060105dc0 2024-09-11T00:18:08.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff065c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff060110a60 con 0x7ff060106780 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060104c60 0x7ff06019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060104c60 0x7ff06019b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33456/0 (socket says 172.21.15.103:33456) 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 -- 172.21.15.103:0/1940648110 learned_addr learned my addr 172.21.15.103:0/1940648110 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff056000640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105dc0 0x7ff06019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05f400640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff060106780 0x7ff06019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 -- 172.21.15.103:0/1940648110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff060106780 msgr2=0x7ff06019c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff060106780 0x7ff06019c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 -- 172.21.15.103:0/1940648110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105dc0 msgr2=0x7ff06019b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105dc0 0x7ff06019b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.211+0000 7ff05ea00640 1 -- 172.21.15.103:0/1940648110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06019cbb0 con 0x7ff060104c60 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.212+0000 7ff05ea00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060104c60 0x7ff06019b080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff05000ea60 tx=0x7ff05000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.212+0000 7ff057e00640 1 -- 172.21.15.103:0/1940648110 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05000cde0 con 0x7ff060104c60 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.212+0000 7ff057e00640 1 -- 172.21.15.103:0/1940648110 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0500035f0 con 0x7ff060104c60 2024-09-11T00:18:08.215 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.212+0000 7ff057e00640 1 -- 172.21.15.103:0/1940648110 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050010690 con 0x7ff060104c60 2024-09-11T00:18:08.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.212+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06019cea0 con 0x7ff060104c60 2024-09-11T00:18:08.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.212+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601a3580 con 0x7ff060104c60 2024-09-11T00:18:08.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.213+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff060105060 con 0x7ff060104c60 2024-09-11T00:18:08.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.216+0000 7ff057e00640 1 -- 172.21.15.103:0/1940648110 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7ff050004000 con 0x7ff060104c60 2024-09-11T00:18:08.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.217+0000 7ff057e00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0440b5360 0x7ff0440b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:08.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.217+0000 7ff057e00640 1 -- 172.21.15.103:0/1940648110 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5829+0+0 (secure 0 0 0) 0x7ff0500e5700 con 0x7ff060104c60 2024-09-11T00:18:08.219 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.217+0000 7ff056000640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0440b5360 0x7ff0440b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:08.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.218+0000 7ff056000640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0440b5360 0x7ff0440b7820 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff048009bd0 tx=0x7ff048005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:08.221 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.219+0000 7ff057e00640 1 -- 172.21.15.103:0/1940648110 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff050056220 con 0x7ff060104c60 2024-09-11T00:18:08.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:08.412+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff060063590 con 0x7ff0440b5360 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] boot 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:18:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:08.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:18:08.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:18:08.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] boot 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:18:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:18:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:18:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: osd.8 [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] boot 2024-09-11T00:18:08.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: osdmap e53: 9 total, 9 up, 9 in 2024-09-11T00:18:08.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-11T00:18:08.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: osdmap e54: 9 total, 9 up, 9 in 2024-09-11T00:18:08.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-11T00:18:08.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:18:08.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:18:08.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:10.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:10 smithi103 ceph-mon[30578]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:18:10.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:10 smithi103 ceph-mon[30578]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T00:18:10.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:10 smithi117 ceph-mon[35525]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:18:10.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:10 smithi117 ceph-mon[35525]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T00:18:10.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:10 smithi183 ceph-mon[35934]: from='client.14487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:18:10.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:10 smithi183 ceph-mon[35934]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T00:18:11.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:11 smithi103 ceph-mon[30578]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:11.338 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:11 smithi117 ceph-mon[35525]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:11.432 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:11 smithi183 ceph-mon[35934]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:12.374 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:12 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/200612150' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"} : dispatch 2024-09-11T00:18:12.374 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:12 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"} : dispatch 2024-09-11T00:18:12.374 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:12 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"}]': finished 2024-09-11T00:18:12.374 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:12 smithi103 ceph-mon[30578]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:18:12.374 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:12.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:12 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/200612150' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"} : dispatch 2024-09-11T00:18:12.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:12 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"} : dispatch 2024-09-11T00:18:12.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:12 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"}]': finished 2024-09-11T00:18:12.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:12 smithi117 ceph-mon[35525]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:18:12.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:12.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:12 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/200612150' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"} : dispatch 2024-09-11T00:18:12.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:12 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"} : dispatch 2024-09-11T00:18:12.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:12 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7999315b-3589-4be7-9d06-4436f21e3235"}]': finished 2024-09-11T00:18:12.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:12 smithi183 ceph-mon[35934]: osdmap e56: 10 total, 9 up, 10 in 2024-09-11T00:18:12.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:13.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:13 smithi103 ceph-mon[30578]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:13.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:13 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3988276617' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:18:13.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:13 smithi117 ceph-mon[35525]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:13.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:13 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3988276617' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:18:13.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:13 smithi183 ceph-mon[35934]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:13.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:13 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3988276617' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:18:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:14 smithi117 ceph-mon[35525]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:18:14.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:14 smithi183 ceph-mon[35934]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:18:14.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:14 smithi103 ceph-mon[30578]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:18:16.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:16 smithi103 ceph-mon[30578]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:18:16.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:16 smithi117 ceph-mon[35525]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:18:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:16 smithi183 ceph-mon[35934]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:18:18.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:18 smithi103 ceph-mon[30578]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:18:18.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:18 smithi117 ceph-mon[35525]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:18:18.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:18 smithi183 ceph-mon[35934]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:18:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:20 smithi117 ceph-mon[35525]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:18:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:20 smithi183 ceph-mon[35934]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:18:20.910 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:20 smithi103 ceph-mon[30578]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T00:18:21.565 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:18:21.565 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:21.565 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:21 smithi103 ceph-mon[30578]: Deploying daemon osd.9 on smithi103 2024-09-11T00:18:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:18:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:21.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:21 smithi117 ceph-mon[35525]: Deploying daemon osd.9 on smithi103 2024-09-11T00:18:21.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-11T00:18:21.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:21.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:21 smithi183 ceph-mon[35934]: Deploying daemon osd.9 on smithi103 2024-09-11T00:18:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:22 smithi117 ceph-mon[35525]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:18:22.730 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:22 smithi183 ceph-mon[35934]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:18:22.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:22 smithi103 ceph-mon[30578]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:18:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:24 smithi103 ceph-mon[30578]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:18:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:24 smithi117 ceph-mon[35525]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:18:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:24 smithi183 ceph-mon[35934]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:18:26.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:26 smithi103 ceph-mon[30578]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:26 smithi117 ceph-mon[35525]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:26 smithi183 ceph-mon[35934]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:28 smithi117 ceph-mon[35525]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:28 smithi183 ceph-mon[35934]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:28.811 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:28 smithi103 ceph-mon[30578]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:29.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:29.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:29.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:29.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:30.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:30 smithi103 ceph-mon[30578]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:30 smithi183 ceph-mon[35934]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:30 smithi117 ceph-mon[35525]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:32.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:32 smithi103 ceph-mon[30578]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:32 smithi117 ceph-mon[35525]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:32 smithi183 ceph-mon[35934]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:34.194 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 9 on host 'smithi103' 2024-09-11T00:18:34.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.191+0000 7ff057e00640 1 -- 172.21.15.103:0/1940648110 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff060063590 con 0x7ff0440b5360 2024-09-11T00:18:34.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0440b5360 msgr2=0x7ff0440b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:34.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0440b5360 0x7ff0440b7820 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff048009bd0 tx=0x7ff048005f90 comp rx=0 tx=0).stop 2024-09-11T00:18:34.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060104c60 msgr2=0x7ff06019b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:34.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060104c60 0x7ff06019b080 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff05000ea60 tx=0x7ff05000ef20 comp rx=0 tx=0).stop 2024-09-11T00:18:34.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 shutdown_connections 2024-09-11T00:18:34.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0440b5360 0x7ff0440b7820 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:34.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff060106780 0x7ff06019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:34.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff060105dc0 0x7ff06019b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:34.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 --2- 172.21.15.103:0/1940648110 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff060104c60 0x7ff06019b080 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:34.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 >> 172.21.15.103:0/1940648110 conn(0x7ff060100510 msgr2=0x7ff060109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:34.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 shutdown_connections 2024-09-11T00:18:34.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:18:34.194+0000 7ff065c00640 1 -- 172.21.15.103:0/1940648110 wait complete. 2024-09-11T00:18:34.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:34 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:34.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:34.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:34 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:34.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:34.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:34.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:34 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:35.075 DEBUG:teuthology.orchestra.run.smithi103:osd.9> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T00:18:35.079 INFO:tasks.cephadm:Deploying osd.10 on smithi117 with /dev/vg_nvme/lv_1... 2024-09-11T00:18:35.079 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:18:35.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:35.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:35.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:35.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:35 smithi103 ceph-mon[30578]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:35.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:35 smithi103 ceph-mon[30578]: from='osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:18:35.318 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:35 smithi103 ceph-mon[30578]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:18:35.336 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:18:35.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:35.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:35.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:35.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:35 smithi117 ceph-mon[35525]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:35.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:35 smithi117 ceph-mon[35525]: from='osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:18:35.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:35 smithi117 ceph-mon[35525]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:18:35.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:35.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:35.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:35.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:35 smithi183 ceph-mon[35934]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:35.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:35 smithi183 ceph-mon[35934]: from='osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:18:35.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:35 smithi183 ceph-mon[35934]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-11T00:18:36.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:36 smithi117 ceph-mon[35525]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:18:36.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:36 smithi117 ceph-mon[35525]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:18:36.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:36 smithi117 ceph-mon[35525]: from='osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:18:36.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:36 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:36.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:36 smithi117 ceph-mon[35525]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:18:36.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:36 smithi183 ceph-mon[35934]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:18:36.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:36 smithi183 ceph-mon[35934]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:18:36.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:36 smithi183 ceph-mon[35934]: from='osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:18:36.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:36 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:36.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:36 smithi183 ceph-mon[35934]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:18:36.594 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:36 smithi103 ceph-mon[30578]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T00:18:36.594 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:36 smithi103 ceph-mon[30578]: osdmap e57: 10 total, 9 up, 10 in 2024-09-11T00:18:36.594 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:36 smithi103 ceph-mon[30578]: from='osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:18:36.595 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:36 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:36.595 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:36 smithi103 ceph-mon[30578]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]} : dispatch 2024-09-11T00:18:36.595 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:18:36 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:18:36.217+0000 7f2387e00640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T00:18:36.595 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:18:36 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:18:36.230+0000 7f2382e00640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:18:37.263 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:37 smithi103 ceph-mon[30578]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:18:37.264 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:37 smithi103 ceph-mon[30578]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T00:18:37.264 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:37 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:37.264 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:37 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:37.264 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:37 smithi103 ceph-mon[30578]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:37.459 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:37 smithi117 ceph-mon[35525]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:18:37.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:37 smithi117 ceph-mon[35525]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T00:18:37.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:37 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:37.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:37 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:37.460 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:37 smithi117 ceph-mon[35525]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:37.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:37 smithi183 ceph-mon[35934]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-09-11T00:18:37.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:37 smithi183 ceph-mon[35934]: osdmap e58: 10 total, 9 up, 10 in 2024-09-11T00:18:37.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:37 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:37.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:37 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:37.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:37 smithi183 ceph-mon[35934]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-11T00:18:38.083 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-11T00:18:38.109 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-09-11T00:18:38.358 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:18:38.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:38 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:18:38.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:38 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:18:38.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:38 smithi117 ceph-mon[35525]: osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] boot 2024-09-11T00:18:38.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:38 smithi117 ceph-mon[35525]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:18:38.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:38.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:38 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:18:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:38 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:18:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:38 smithi183 ceph-mon[35934]: osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] boot 2024-09-11T00:18:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:38 smithi183 ceph-mon[35934]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:18:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:38.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:38 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:18:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:38 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:18:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:38 smithi103 ceph-mon[30578]: osd.9 [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] boot 2024-09-11T00:18:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:38 smithi103 ceph-mon[30578]: osdmap e59: 10 total, 10 up, 10 in 2024-09-11T00:18:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 -- 172.21.15.117:0/2446248997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230103b90 msgr2=0x7f6230104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 --2- 172.21.15.117:0/2446248997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230103b90 0x7f6230104010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6224009930 tx=0x7f622402a420 comp rx=0 tx=0).stop 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 -- 172.21.15.117:0/2446248997 shutdown_connections 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 --2- 172.21.15.117:0/2446248997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6230104550 0x7f623010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 --2- 172.21.15.117:0/2446248997 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230103b90 0x7f6230104010 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 --2- 172.21.15.117:0/2446248997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6230102a30 0x7f6230102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 -- 172.21.15.117:0/2446248997 >> 172.21.15.117:0/2446248997 conn(0x7f62300fe320 msgr2=0x7f6230100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 -- 172.21.15.117:0/2446248997 shutdown_connections 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.823+0000 7f6235f35640 1 -- 172.21.15.117:0/2446248997 wait complete. 2024-09-11T00:18:38.851 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 Processor -- start 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 -- start start 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6230102a30 0x7f6230198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6230103b90 0x7f6230199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230104550 0x7f623019a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f623010e6b0 con 0x7f6230102a30 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f623010e530 con 0x7f6230104550 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f6235f35640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f623010e830 con 0x7f6230103b90 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f622ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230104550 0x7f623019a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f622ffff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230104550 0x7f623019a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:34970/0 (socket says 172.21.15.117:34970) 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f622ffff640 1 -- 172.21.15.117:0/4203433832 learned_addr learned my addr 172.21.15.117:0/4203433832 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.824+0000 7f622f7fe640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6230102a30 0x7f6230198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:38.852 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622effd640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6230103b90 0x7f6230199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622ffff640 1 -- 172.21.15.117:0/4203433832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6230103b90 msgr2=0x7f6230199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622ffff640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6230103b90 0x7f6230199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622ffff640 1 -- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6230102a30 msgr2=0x7f6230198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622ffff640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6230102a30 0x7f6230198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622ffff640 1 -- 172.21.15.117:0/4203433832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623019a900 con 0x7f6230104550 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622effd640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6230103b90 0x7f6230199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622ffff640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230104550 0x7f623019a060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f622800eaa0 tx=0x7f622800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622cff9640 1 -- 172.21.15.117:0/4203433832 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622800cdb0 con 0x7f6230104550 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623019abf0 con 0x7f6230104550 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622cff9640 1 -- 172.21.15.117:0/4203433832 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6228005020 con 0x7f6230104550 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f622cff9640 1 -- 172.21.15.117:0/4203433832 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62280116d0 con 0x7f6230104550 2024-09-11T00:18:38.853 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.825+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62301a12d0 con 0x7f6230104550 2024-09-11T00:18:38.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.826+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6230109d50 con 0x7f6230104550 2024-09-11T00:18:38.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.827+0000 7f622cff9640 1 -- 172.21.15.117:0/4203433832 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f622800f040 con 0x7f6230104550 2024-09-11T00:18:38.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.828+0000 7f622cff9640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62180b52e0 0x7f62180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:18:38.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.828+0000 7f622cff9640 1 -- 172.21.15.117:0/4203433832 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7f62280e5610 con 0x7f6230104550 2024-09-11T00:18:38.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.828+0000 7f622f7fe640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62180b52e0 0x7f62180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:18:38.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.829+0000 7f622f7fe640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62180b52e0 0x7f62180b77a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f622000bbe0 tx=0x7f622000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:18:38.854 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:38.831+0000 7f622cff9640 1 -- 172.21.15.117:0/4203433832 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6228016030 con 0x7f6230104550 2024-09-11T00:18:39.019 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:39.017+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6230063590 con 0x7f62180b52e0 2024-09-11T00:18:39.423 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:39 smithi117 ceph-mon[35525]: osdmap e60: 10 total, 10 up, 10 in 2024-09-11T00:18:39.423 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:39 smithi117 ceph-mon[35525]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:39.424 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:18:39.424 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:18:39.424 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:39 smithi183 ceph-mon[35934]: osdmap e60: 10 total, 10 up, 10 in 2024-09-11T00:18:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:39 smithi183 ceph-mon[35934]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:18:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:18:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:39.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:39 smithi103 ceph-mon[30578]: osdmap e60: 10 total, 10 up, 10 in 2024-09-11T00:18:39.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:39 smithi103 ceph-mon[30578]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:39.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:18:39.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:18:39.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:40.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='client.24488 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:18:40.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi103 2024-09-11T00:18:40.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-11T00:18:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:40.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='client.24488 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi103 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='client.24488 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi103 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi103 to 4272M 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:41 smithi117 ceph-mon[35525]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:41.601 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:41 smithi103 ceph-mon[30578]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:41.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:41 smithi183 ceph-mon[35934]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:42 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/2773687939' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"} : dispatch 2024-09-11T00:18:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:42 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"} : dispatch 2024-09-11T00:18:42.345 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:42 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"}]': finished 2024-09-11T00:18:42.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:42 smithi103 ceph-mon[30578]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:18:42.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:18:42.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:42 smithi103 ceph-mon[30578]: from='client.? 172.21.15.117:0/279828026' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:18:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:42 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/2773687939' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"} : dispatch 2024-09-11T00:18:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:42 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"} : dispatch 2024-09-11T00:18:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:42 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"}]': finished 2024-09-11T00:18:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:42 smithi117 ceph-mon[35525]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:18:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:18:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:42 smithi117 ceph-mon[35525]: from='client.? 172.21.15.117:0/279828026' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:18:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:42 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/2773687939' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"} : dispatch 2024-09-11T00:18:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:42 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"} : dispatch 2024-09-11T00:18:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:42 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bde15347-8049-49ab-a4e7-06a5cba2d225"}]': finished 2024-09-11T00:18:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:42 smithi183 ceph-mon[35934]: osdmap e61: 11 total, 10 up, 11 in 2024-09-11T00:18:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:18:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:42 smithi183 ceph-mon[35934]: from='client.? 172.21.15.117:0/279828026' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:18:43.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:43 smithi103 ceph-mon[30578]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:43.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:43 smithi117 ceph-mon[35525]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:43.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:43 smithi183 ceph-mon[35934]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:45.597 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:45 smithi117 ceph-mon[35525]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:45.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:45 smithi103 ceph-mon[30578]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:45.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:45 smithi183 ceph-mon[35934]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:47.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:47 smithi103 ceph-mon[30578]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:47.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:47 smithi117 ceph-mon[35525]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:47.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:47 smithi183 ceph-mon[35934]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:49.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:49 smithi103 ceph-mon[30578]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:49.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:18:49.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:49 smithi117 ceph-mon[35525]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:49.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:49 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:18:49.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:49 smithi183 ceph-mon[35934]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:49.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:49 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-11T00:18:50.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:50.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:50 smithi103 ceph-mon[30578]: Deploying daemon osd.10 on smithi117 2024-09-11T00:18:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:50 smithi117 ceph-mon[35525]: Deploying daemon osd.10 on smithi117 2024-09-11T00:18:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:50 smithi183 ceph-mon[35934]: Deploying daemon osd.10 on smithi117 2024-09-11T00:18:51.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:51 smithi103 ceph-mon[30578]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:51 smithi117 ceph-mon[35525]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:51.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:51 smithi183 ceph-mon[35934]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:53 smithi117 ceph-mon[35525]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:53.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:53 smithi103 ceph-mon[30578]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:53 smithi183 ceph-mon[35934]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:55.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:55 smithi103 ceph-mon[30578]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:55.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:55.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:55.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:55.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:55 smithi117 ceph-mon[35525]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:55.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:55.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:55.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:55.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:55 smithi183 ceph-mon[35934]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:55.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:55.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:55.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:57.610 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:57 smithi117 ceph-mon[35525]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:57.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:57 smithi103 ceph-mon[30578]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:57.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:57 smithi183 ceph-mon[35934]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:58.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.636+0000 7f622cff9640 1 -- 172.21.15.117:0/4203433832 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6230063590 con 0x7f62180b52e0 2024-09-11T00:18:58.639 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 10 on host 'smithi117' 2024-09-11T00:18:58.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62180b52e0 msgr2=0x7f62180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:58.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62180b52e0 0x7f62180b77a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f622000bbe0 tx=0x7f622000b390 comp rx=0 tx=0).stop 2024-09-11T00:18:58.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230104550 msgr2=0x7f623019a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:18:58.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230104550 0x7f623019a060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f622800eaa0 tx=0x7f622800ef60 comp rx=0 tx=0).stop 2024-09-11T00:18:58.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 shutdown_connections 2024-09-11T00:18:58.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62180b52e0 0x7f62180b77a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:58.641 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6230104550 0x7f623019a060 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:58.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6230103b90 0x7f6230199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:58.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 --2- 172.21.15.117:0/4203433832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6230102a30 0x7f6230198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:18:58.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 >> 172.21.15.117:0/4203433832 conn(0x7f62300fe320 msgr2=0x7f6230106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:18:58.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 shutdown_connections 2024-09-11T00:18:58.642 INFO:teuthology.orchestra.run.smithi117.stderr:2024-09-11T00:18:58.639+0000 7f6235f35640 1 -- 172.21.15.117:0/4203433832 wait complete. 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:18:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:59.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:18:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:18:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:18:59.307 DEBUG:teuthology.orchestra.run.smithi117:osd.10> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T00:18:59.309 INFO:tasks.cephadm:Deploying osd.11 on smithi183 with /dev/vg_nvme/lv_1... 2024-09-11T00:18:59.309 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T00:18:59.562 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:19:00.478 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:00 smithi117 ceph-mon[35525]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:19:00.728 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:00 smithi183 ceph-mon[35934]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:19:00.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:00 smithi103 ceph-mon[30578]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:19:01.187 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:19:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:19:00.991+0000 7f055ab48640 -1 osd.10 0 log_to_monitors true 2024-09-11T00:19:01.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:01 smithi183 ceph-mon[35934]: from='osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:19:01.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:01 smithi183 ceph-mon[35934]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:19:01.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:01 smithi117 ceph-mon[35525]: from='osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:19:01.785 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:01 smithi117 ceph-mon[35525]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:19:01.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:01 smithi103 ceph-mon[30578]: from='osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:19:01.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:01 smithi103 ceph-mon[30578]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-11T00:19:02.261 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-11T00:19:02.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-09-11T00:19:02.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi117 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:02.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:02.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:02 smithi183 ceph-mon[35934]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:19:02.553 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi117 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:02 smithi103 ceph-mon[30578]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:19:02.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:19:02.456+0000 7f0556acb640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T00:19:02.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:19:02.466+0000 7f05528f2640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:19:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T00:19:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:19:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: osdmap e62: 11 total, 10 up, 11 in 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi117 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi117 to 4272M 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:02.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:02 smithi117 ceph-mon[35525]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 296 MiB used, 894 GiB / 894 GiB avail 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 -- 172.21.15.183:0/2615679349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 msgr2=0x7fc7d0104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 --2- 172.21.15.183:0/2615679349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0104040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8009d00 tx=0x7fc7b802a910 comp rx=0 tx=0).stop 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 -- 172.21.15.183:0/2615679349 shutdown_connections 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 --2- 172.21.15.183:0/2615679349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d0104580 0x7fc7d010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 --2- 172.21.15.183:0/2615679349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0104040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 --2- 172.21.15.183:0/2615679349 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d0102a60 0x7fc7d0102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 -- 172.21.15.183:0/2615679349 >> 172.21.15.183:0/2615679349 conn(0x7fc7d00fe350 msgr2=0x7fc7d0100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 -- 172.21.15.183:0/2615679349 shutdown_connections 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.070+0000 7fc7d7097640 1 -- 172.21.15.183:0/2615679349 wait complete. 2024-09-11T00:19:03.099 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 Processor -- start 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 -- start start 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d0102a60 0x7fc7d0076c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d0104580 0x7fc7d0075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d010e740 con 0x7fc7d0104580 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d010e5c0 con 0x7fc7d0102a60 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d7097640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d010e8c0 con 0x7fc7d0103bc0 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7cffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7cffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:40868/0 (socket says 172.21.15.183:40868) 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7cffff640 1 -- 172.21.15.183:0/3755581937 learned_addr learned my addr 172.21.15.183:0/3755581937 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.071+0000 7fc7d4e33640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d0102a60 0x7fc7d0076c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:03.100 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7d5634640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d0104580 0x7fc7d0075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cffff640 1 -- 172.21.15.183:0/3755581937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d0102a60 msgr2=0x7fc7d0076c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cffff640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d0102a60 0x7fc7d0076c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cffff640 1 -- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d0104580 msgr2=0x7fc7d0075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cffff640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d0104580 0x7fc7d0075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cffff640 1 -- 172.21.15.183:0/3755581937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d0076280 con 0x7fc7d0103bc0 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cffff640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0075540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8003c20 tx=0x7fc7b8003c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cdffb640 1 -- 172.21.15.183:0/3755581937 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80307d0 con 0x7fc7d0103bc0 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d01a2f50 con 0x7fc7d0103bc0 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cdffb640 1 -- 172.21.15.183:0/3755581937 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b80317d0 con 0x7fc7d0103bc0 2024-09-11T00:19:03.101 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7cdffb640 1 -- 172.21.15.183:0/3755581937 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b802fd80 con 0x7fc7d0103bc0 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.072+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d01a3480 con 0x7fc7d0103bc0 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.073+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d0109de0 con 0x7fc7d0103bc0 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.075+0000 7fc7cdffb640 1 -- 172.21.15.183:0/3755581937 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fc7b8004b40 con 0x7fc7d0103bc0 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.075+0000 7fc7cdffb640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7bc0b52e0 0x7fc7bc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.075+0000 7fc7cdffb640 1 -- 172.21.15.183:0/3755581937 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6727+0+0 (secure 0 0 0) 0x7fc7b8106c20 con 0x7fc7d0103bc0 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.075+0000 7fc7d4e33640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7bc0b52e0 0x7fc7bc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.076+0000 7fc7d4e33640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7bc0b52e0 0x7fc7bc0b77a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009890 tx=0x7fc7c8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:03.102 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.078+0000 7fc7cdffb640 1 -- 172.21.15.183:0/3755581937 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc7b80ce110 con 0x7fc7d0103bc0 2024-09-11T00:19:03.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:03.266+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc7d0063590 con 0x7fc7bc0b52e0 2024-09-11T00:19:03.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] boot 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:19:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:03.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] boot 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:19:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-11T00:19:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: osdmap e63: 11 total, 10 up, 11 in 2024-09-11T00:19:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: osd.10 [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] boot 2024-09-11T00:19:03.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: osdmap e64: 11 total, 11 up, 11 in 2024-09-11T00:19:03.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-11T00:19:03.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='client.24508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:19:03.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-11T00:19:03.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-11T00:19:03.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:04.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:04 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:19:04.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:04 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:19:04.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:04 smithi183 ceph-mon[35934]: osdmap e65: 11 total, 11 up, 11 in 2024-09-11T00:19:04.678 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:04 smithi183 ceph-mon[35934]: pgmap v254: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:04.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:04 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:19:04.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:04 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:19:04.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:04 smithi103 ceph-mon[30578]: osdmap e65: 11 total, 11 up, 11 in 2024-09-11T00:19:04.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:04 smithi103 ceph-mon[30578]: pgmap v254: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:04 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:19:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:04 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:19:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:04 smithi117 ceph-mon[35525]: osdmap e65: 11 total, 11 up, 11 in 2024-09-11T00:19:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:04 smithi117 ceph-mon[35525]: pgmap v254: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:06.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:06 smithi183 ceph-mon[35934]: osdmap e66: 11 total, 11 up, 11 in 2024-09-11T00:19:06.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:06 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/1096092162' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"} : dispatch 2024-09-11T00:19:06.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:06 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"} : dispatch 2024-09-11T00:19:06.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:06 smithi183 ceph-mon[35934]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"}]': finished 2024-09-11T00:19:06.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:06 smithi183 ceph-mon[35934]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:19:06.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:06 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:06.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:06 smithi103 ceph-mon[30578]: osdmap e66: 11 total, 11 up, 11 in 2024-09-11T00:19:06.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:06 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/1096092162' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"} : dispatch 2024-09-11T00:19:06.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:06 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"} : dispatch 2024-09-11T00:19:06.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:06 smithi103 ceph-mon[30578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"}]': finished 2024-09-11T00:19:06.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:06 smithi103 ceph-mon[30578]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:19:06.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:06 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:06 smithi117 ceph-mon[35525]: osdmap e66: 11 total, 11 up, 11 in 2024-09-11T00:19:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:06 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/1096092162' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"} : dispatch 2024-09-11T00:19:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:06 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"} : dispatch 2024-09-11T00:19:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:06 smithi117 ceph-mon[35525]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bdface1-a9b3-4175-93ea-db2997b3c55b"}]': finished 2024-09-11T00:19:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:06 smithi117 ceph-mon[35525]: osdmap e67: 12 total, 11 up, 12 in 2024-09-11T00:19:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:06 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:07.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:07 smithi103 ceph-mon[30578]: from='client.? 172.21.15.183:0/4102788122' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:19:07.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:07 smithi103 ceph-mon[30578]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:07.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:07 smithi117 ceph-mon[35525]: from='client.? 172.21.15.183:0/4102788122' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:19:07.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:07 smithi117 ceph-mon[35525]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:07.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:07 smithi183 ceph-mon[35934]: from='client.? 172.21.15.183:0/4102788122' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-11T00:19:07.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:07 smithi183 ceph-mon[35934]: pgmap v257: 1 pgs: 1 peering; 577 KiB data, 323 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:08 smithi183 ceph-mon[35934]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-11T00:19:08.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:08 smithi103 ceph-mon[30578]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-11T00:19:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:08 smithi117 ceph-mon[35525]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-09-11T00:19:10.761 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:10 smithi183 ceph-mon[35934]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:19:10.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:10 smithi103 ceph-mon[30578]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:19:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:10 smithi117 ceph-mon[35525]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-11T00:19:12.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:12 smithi183 ceph-mon[35934]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:19:12.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:12 smithi103 ceph-mon[30578]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:19:12.739 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:12 smithi117 ceph-mon[35525]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T00:19:14.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:14 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:19:14.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:14 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:14.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:14 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:19:14.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:14 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:14.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:14 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-11T00:19:14.443 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:14 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:15.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:15 smithi103 ceph-mon[30578]: Deploying daemon osd.11 on smithi183 2024-09-11T00:19:15.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:15 smithi103 ceph-mon[30578]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:19:15.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:15 smithi117 ceph-mon[35525]: Deploying daemon osd.11 on smithi183 2024-09-11T00:19:15.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:15 smithi117 ceph-mon[35525]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:19:15.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:15 smithi183 ceph-mon[35934]: Deploying daemon osd.11 on smithi183 2024-09-11T00:19:15.469 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:15 smithi183 ceph-mon[35934]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-11T00:19:16.744 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:16 smithi183 ceph-mon[35934]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:19:16.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:16 smithi103 ceph-mon[30578]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:19:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:16 smithi117 ceph-mon[35525]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-11T00:19:18.666 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:18 smithi183 ceph-mon[35934]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:19:18.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:18 smithi103 ceph-mon[30578]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:19:18.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:18 smithi117 ceph-mon[35525]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T00:19:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:20.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:20 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:20.420 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:20.420 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:20.420 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:20 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:20 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:21.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:21 smithi103 ceph-mon[30578]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:21 smithi117 ceph-mon[35525]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:21.486 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:21 smithi183 ceph-mon[35934]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:22.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:22 smithi183 ceph-mon[35934]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:22.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:22 smithi103 ceph-mon[30578]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:22 smithi117 ceph-mon[35525]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:23.509 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.506+0000 7fc7cdffb640 1 -- 172.21.15.183:0/3755581937 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc7d0063590 con 0x7fc7bc0b52e0 2024-09-11T00:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 11 on host 'smithi183' 2024-09-11T00:19:23.512 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.509+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7bc0b52e0 msgr2=0x7fc7bc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:23.512 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.509+0000 7fc7d7097640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7bc0b52e0 0x7fc7bc0b77a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009890 tx=0x7fc7c8008040 comp rx=0 tx=0).stop 2024-09-11T00:19:23.512 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.509+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 msgr2=0x7fc7d0075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:23.512 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.509+0000 7fc7d7097640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0075540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8003c20 tx=0x7fc7b8003c50 comp rx=0 tx=0).stop 2024-09-11T00:19:23.512 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 shutdown_connections 2024-09-11T00:19:23.513 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7bc0b52e0 0x7fc7bc0b77a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:23.513 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7d0104580 0x7fc7d0075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:23.513 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7d0103bc0 0x7fc7d0075540 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:23.513 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 --2- 172.21.15.183:0/3755581937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7d0102a60 0x7fc7d0076c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:23.513 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 >> 172.21.15.183:0/3755581937 conn(0x7fc7d00fe350 msgr2=0x7fc7d0106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:23.513 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 shutdown_connections 2024-09-11T00:19:23.513 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:23.510+0000 7fc7d7097640 1 -- 172.21.15.183:0/3755581937 wait complete. 2024-09-11T00:19:24.275 DEBUG:teuthology.orchestra.run.smithi183:osd.11> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T00:19:24.277 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T00:19:24.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T00:19:24.660 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:24 smithi103 ceph-mon[30578]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:24 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:24 smithi183 ceph-mon[35934]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:24 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:24 smithi117 ceph-mon[35525]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:24 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:25.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 -- 172.21.15.103:0/1218894923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc1065c0 msgr2=0x7f72cc10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:25.227 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 --2- 172.21.15.103:0/1218894923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc1065c0 0x7f72cc10cf40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009cd0 tx=0x7f72c402a8f0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 -- 172.21.15.103:0/1218894923 shutdown_connections 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 --2- 172.21.15.103:0/1218894923 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc1065c0 0x7f72cc10cf40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 --2- 172.21.15.103:0/1218894923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc105c00 0x7f72cc106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 --2- 172.21.15.103:0/1218894923 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc104aa0 0x7f72cc104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 -- 172.21.15.103:0/1218894923 >> 172.21.15.103:0/1218894923 conn(0x7f72cc100370 msgr2=0x7f72cc1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 -- 172.21.15.103:0/1218894923 shutdown_connections 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.224+0000 7f72d1e00640 1 -- 172.21.15.103:0/1218894923 wait complete. 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 Processor -- start 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 -- start start 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc104aa0 0x7f72cc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc105c00 0x7f72cc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:25.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc1065c0 0x7f72cc19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc110720 con 0x7f72cc105c00 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1105a0 con 0x7f72cc104aa0 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.225+0000 7f72d1e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc1108a0 con 0x7f72cc1065c0 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc104aa0 0x7f72cc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72cbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc105c00 0x7f72cc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc1065c0 0x7f72cc19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc104aa0 0x7f72cc19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:34090/0 (socket says 172.21.15.103:34090) 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc1065c0 0x7f72cc19bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:43796/0 (socket says 172.21.15.103:43796) 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72cbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc105c00 0x7f72cc19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44400/0 (socket says 172.21.15.103:44400) 2024-09-11T00:19:25.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d0a00640 1 -- 172.21.15.103:0/3133719810 learned_addr learned my addr 172.21.15.103:0/3133719810 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:25.230 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 -- 172.21.15.103:0/3133719810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc104aa0 msgr2=0x7f72cc19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc104aa0 0x7f72cc19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 -- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc105c00 msgr2=0x7f72cc19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc105c00 0x7f72cc19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 -- 172.21.15.103:0/3133719810 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc19c860 con 0x7f72cc1065c0 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72d1400640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc1065c0 0x7f72cc19bfc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72c402ae10 tx=0x7f72c4003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.226+0000 7f72c9600640 1 -- 172.21.15.103:0/3133719810 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4004240 con 0x7f72cc1065c0 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.227+0000 7f72c9600640 1 -- 172.21.15.103:0/3133719810 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72c4040070 con 0x7f72cc1065c0 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.227+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc19caf0 con 0x7f72cc1065c0 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.227+0000 7f72c9600640 1 -- 172.21.15.103:0/3133719810 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4045ca0 con 0x7f72cc1065c0 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.227+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc1a3230 con 0x7f72cc1065c0 2024-09-11T00:19:25.231 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.228+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72cc104ea0 con 0x7f72cc1065c0 2024-09-11T00:19:25.232 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.230+0000 7f72c9600640 1 -- 172.21.15.103:0/3133719810 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f72c40043e0 con 0x7f72cc1065c0 2024-09-11T00:19:25.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.231+0000 7f72c9600640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f72b40b5390 0x7f72b40b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:25.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.231+0000 7f72c9600640 1 -- 172.21.15.103:0/3133719810 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6809+0+0 (secure 0 0 0) 0x7f72c4106840 con 0x7f72cc1065c0 2024-09-11T00:19:25.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.231+0000 7f72d0a00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f72b40b5390 0x7f72b40b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:25.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.232+0000 7f72d0a00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f72b40b5390 0x7f72b40b7850 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009c90 tx=0x7f72c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:25.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.235+0000 7f72c9600640 1 -- 172.21.15.103:0/3133719810 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f72c40cdc40 con 0x7f72cc1065c0 2024-09-11T00:19:25.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.421+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f72cc063d50 con 0x7f72cc1065c0 2024-09-11T00:19:25.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.422+0000 7f72c9600640 1 -- 172.21.15.103:0/3133719810 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v67) ==== 74+0+133 (secure 0 0 0) 0x7f72c40cd540 con 0x7f72cc1065c0 2024-09-11T00:19:25.424 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:25.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f72b40b5390 msgr2=0x7f72b40b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:25.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f72b40b5390 0x7f72b40b7850 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009c90 tx=0x7f72c0005f90 comp rx=0 tx=0).stop 2024-09-11T00:19:25.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc1065c0 msgr2=0x7f72cc19bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc1065c0 0x7f72cc19bfc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72c402ae10 tx=0x7f72c4003ba0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 shutdown_connections 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f72b40b5390 0x7f72b40b7850 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72cc1065c0 0x7f72cc19bfc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72cc105c00 0x7f72cc19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 --2- 172.21.15.103:0/3133719810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72cc104aa0 0x7f72cc19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 >> 172.21.15.103:0/3133719810 conn(0x7f72cc100370 msgr2=0x7f72cc108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 shutdown_connections 2024-09-11T00:19:25.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:25.424+0000 7f72d1e00640 1 -- 172.21.15.103:0/3133719810 wait complete. 2024-09-11T00:19:25.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:25 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3133719810' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:19:25.816 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:25 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3133719810' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:19:25.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:25 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3133719810' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:19:26.280 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:19:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:19:26.044+0000 7f87c93c9640 -1 osd.11 0 log_to_monitors true 2024-09-11T00:19:26.349 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":67,"num_osds":12,"num_up_osds":11,"osd_up_since":1726013943,"num_in_osds":12,"osd_in_since":1726013945,"num_remapped_pgs":0} 2024-09-11T00:19:26.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:26 smithi183 ceph-mon[35934]: from='osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:19:26.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:26 smithi183 ceph-mon[35934]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:19:26.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:26 smithi183 ceph-mon[35934]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:26.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:26 smithi103 ceph-mon[30578]: from='osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:19:26.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:26 smithi103 ceph-mon[30578]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:19:26.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:26 smithi103 ceph-mon[30578]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:26 smithi117 ceph-mon[35525]: from='osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:19:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:26 smithi117 ceph-mon[35525]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-11T00:19:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:26 smithi117 ceph-mon[35525]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:27.350 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T00:19:27.639 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:27 smithi183 ceph-mon[35934]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:19:27.639 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:27 smithi183 ceph-mon[35934]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T00:19:27.639 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:27.639 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:27 smithi183 ceph-mon[35934]: from='osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:19:27.640 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:27 smithi183 ceph-mon[35934]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:19:27.640 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:19:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:19:27.530+0000 7f87c534c640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T00:19:27.640 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:19:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:19:27.544+0000 7f87c1173640 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T00:19:27.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:27 smithi103 ceph-mon[30578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:19:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:27 smithi103 ceph-mon[30578]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T00:19:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:27 smithi103 ceph-mon[30578]: from='osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:19:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:27 smithi103 ceph-mon[30578]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:19:27.734 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:27 smithi117 ceph-mon[35525]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T00:19:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:27 smithi117 ceph-mon[35525]: osdmap e68: 12 total, 11 up, 12 in 2024-09-11T00:19:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:27 smithi117 ceph-mon[35525]: from='osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:19:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:27 smithi117 ceph-mon[35525]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-09-11T00:19:28.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 -- 172.21.15.103:0/3259455993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 msgr2=0x7f760c10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:28.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 --2- 172.21.15.103:0/3259455993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 0x7f760c10ac50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7604009900 tx=0x7f760402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 -- 172.21.15.103:0/3259455993 shutdown_connections 2024-09-11T00:19:28.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 --2- 172.21.15.103:0/3259455993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 0x7f760c10ac50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 --2- 172.21.15.103:0/3259455993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f760c100fc0 0x7f760c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 --2- 172.21.15.103:0/3259455993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 0x7f760c100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 -- 172.21.15.103:0/3259455993 >> 172.21.15.103:0/3259455993 conn(0x7f760c0fc530 msgr2=0x7f760c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:28.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.344+0000 7f7614600640 1 -- 172.21.15.103:0/3259455993 shutdown_connections 2024-09-11T00:19:28.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.345+0000 7f7614600640 1 -- 172.21.15.103:0/3259455993 wait complete. 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 Processor -- start 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 -- start start 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 0x7f760c19af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f760c100fc0 0x7f760c19b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 0x7f760c19c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10e680 con 0x7f760c108790 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10e500 con 0x7f760c100680 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.346+0000 7f7614600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f760c10e800 con 0x7f760c100fc0 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 0x7f760c19af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:28.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 0x7f760c19af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:34106/0 (socket says 172.21.15.103:34106) 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 -- 172.21.15.103:0/987065999 learned_addr learned my addr 172.21.15.103:0/987065999 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7612600640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 0x7f760c19c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611200640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f760c100fc0 0x7f760c19b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 -- 172.21.15.103:0/987065999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f760c100fc0 msgr2=0x7f760c19b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f760c100fc0 0x7f760c19b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 -- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 msgr2=0x7f760c19c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 0x7f760c19c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 -- 172.21.15.103:0/987065999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c19ca80 con 0x7f760c100680 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.347+0000 7f7611c00640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 0x7f760c19af50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f760000eab0 tx=0x7f760000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.348+0000 7f760a000640 1 -- 172.21.15.103:0/987065999 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760000ce40 con 0x7f760c100680 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.348+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c19cd70 con 0x7f760c100680 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.348+0000 7f760a000640 1 -- 172.21.15.103:0/987065999 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7600011920 con 0x7f760c100680 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.348+0000 7f760a000640 1 -- 172.21.15.103:0/987065999 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600010a30 con 0x7f760c100680 2024-09-11T00:19:28.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.348+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c1a3450 con 0x7f760c100680 2024-09-11T00:19:28.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.349+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c1a3680 con 0x7f760c100680 2024-09-11T00:19:28.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.351+0000 7f760a000640 1 -- 172.21.15.103:0/987065999 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f760000f040 con 0x7f760c100680 2024-09-11T00:19:28.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.353+0000 7f760a000640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f75f40b52e0 0x7f75f40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:28.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.353+0000 7f760a000640 1 -- 172.21.15.103:0/987065999 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6847+0+0 (secure 0 0 0) 0x7f76000e5920 con 0x7f760c100680 2024-09-11T00:19:28.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.353+0000 7f7611200640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f75f40b52e0 0x7f75f40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:28.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.354+0000 7f7611200640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f75f40b52e0 0x7f75f40b77a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009bd0 tx=0x7f75f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:28.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.354+0000 7f760a000640 1 -- 172.21.15.103:0/987065999 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7600016030 con 0x7f760c100680 2024-09-11T00:19:28.547 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.546+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f760c1a3850 con 0x7f760c100680 2024-09-11T00:19:28.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.547+0000 7f760a000640 1 -- 172.21.15.103:0/987065999 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f76000acd00 con 0x7f760c100680 2024-09-11T00:19:28.548 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:28.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f75f40b52e0 msgr2=0x7f75f40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:28.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f75f40b52e0 0x7f75f40b77a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009bd0 tx=0x7f75f8005f90 comp rx=0 tx=0).stop 2024-09-11T00:19:28.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 msgr2=0x7f760c19af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:28.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 0x7f760c19af50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f760000eab0 tx=0x7f760000ef70 comp rx=0 tx=0).stop 2024-09-11T00:19:28.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 shutdown_connections 2024-09-11T00:19:28.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f75f40b52e0 0x7f75f40b77a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f760c108790 0x7f760c19c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f760c100fc0 0x7f760c19b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 --2- 172.21.15.103:0/987065999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f760c100680 0x7f760c19af50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:28.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.549+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 >> 172.21.15.103:0/987065999 conn(0x7f760c0fc530 msgr2=0x7f760c0fc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:28.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.550+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 shutdown_connections 2024-09-11T00:19:28.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:28.550+0000 7f7614600640 1 -- 172.21.15.103:0/987065999 wait complete. 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: osdmap e69: 12 total, 11 up, 12 in 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: Detected new or changed devices on smithi183 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:28 smithi103 ceph-mon[30578]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:19:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: osdmap e69: 12 total, 11 up, 12 in 2024-09-11T00:19:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: Detected new or changed devices on smithi183 2024-09-11T00:19:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:28.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:28.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:28.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:28.939 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:28 smithi117 ceph-mon[35525]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-11T00:19:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: osdmap e69: 12 total, 11 up, 12 in 2024-09-11T00:19:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: Detected new or changed devices on smithi183 2024-09-11T00:19:29.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:29.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:29.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:29.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:29.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:29.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:19:29.033 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-11T00:19:29.033 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:29.033 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:29.033 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:29.033 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:29.034 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:29.034 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:29.034 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:29.034 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:29.034 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:28 smithi183 ceph-mon[35934]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-11T00:19:29.477 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726013968,"num_in_osds":12,"osd_in_since":1726013945,"num_remapped_pgs":0} 2024-09-11T00:19:29.477 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:19:29.845 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:29.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:29 smithi103 ceph-mon[30578]: purged_snaps scrub starts 2024-09-11T00:19:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:29 smithi103 ceph-mon[30578]: purged_snaps scrub ok 2024-09-11T00:19:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:29 smithi103 ceph-mon[30578]: osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] boot 2024-09-11T00:19:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:29 smithi103 ceph-mon[30578]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:19:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:29 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:29 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/987065999' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:19:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:29 smithi117 ceph-mon[35525]: purged_snaps scrub starts 2024-09-11T00:19:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:29 smithi117 ceph-mon[35525]: purged_snaps scrub ok 2024-09-11T00:19:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:29 smithi117 ceph-mon[35525]: osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] boot 2024-09-11T00:19:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:29 smithi117 ceph-mon[35525]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:19:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:29 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:29 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/987065999' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:19:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:29 smithi183 ceph-mon[35934]: purged_snaps scrub starts 2024-09-11T00:19:30.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:29 smithi183 ceph-mon[35934]: purged_snaps scrub ok 2024-09-11T00:19:30.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:29 smithi183 ceph-mon[35934]: osd.11 [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] boot 2024-09-11T00:19:30.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:29 smithi183 ceph-mon[35934]: osdmap e70: 12 total, 12 up, 12 in 2024-09-11T00:19:30.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:29 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-11T00:19:30.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:29 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/987065999' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-11T00:19:30.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 -- 172.21.15.103:0/2915003133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 msgr2=0x7f908810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 --2- 172.21.15.103:0/2915003133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 0x7f908810f930 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9080009ca0 tx=0x7f908002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 -- 172.21.15.103:0/2915003133 shutdown_connections 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 --2- 172.21.15.103:0/2915003133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 0x7f908810f930 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 --2- 172.21.15.103:0/2915003133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90881087b0 0x7f9088108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 --2- 172.21.15.103:0/2915003133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9088102da0 0x7f9088103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 -- 172.21.15.103:0/2915003133 >> 172.21.15.103:0/2915003133 conn(0x7f90880fe810 msgr2=0x7f9088100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 -- 172.21.15.103:0/2915003133 shutdown_connections 2024-09-11T00:19:30.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.402+0000 7f908f400640 1 -- 172.21.15.103:0/2915003133 wait complete. 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 Processor -- start 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 -- start start 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9088102da0 0x7f9088199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90881087b0 0x7f9088199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 0x7f908819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9088112e40 con 0x7f9088109130 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9088112cc0 con 0x7f9088102da0 2024-09-11T00:19:30.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.404+0000 7f908f400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9088112fc0 con 0x7f90881087b0 2024-09-11T00:19:30.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9088102da0 0x7f9088199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9088102da0 0x7f9088199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:34126/0 (socket says 172.21.15.103:34126) 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f9087e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90881087b0 0x7f9088199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 -- 172.21.15.103:0/377777726 learned_addr learned my addr 172.21.15.103:0/377777726 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908d400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 0x7f908819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 -- 172.21.15.103:0/377777726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90881087b0 msgr2=0x7f9088199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90881087b0 0x7f9088199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 -- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 msgr2=0x7f908819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 0x7f908819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 -- 172.21.15.103:0/377777726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908819f2c0 con 0x7f9088102da0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908d400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 0x7f908819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.405+0000 7f908ca00640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9088102da0 0x7f9088199210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9f0 tx=0x7f907c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.406+0000 7f9085600640 1 -- 172.21.15.103:0/377777726 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c00cd80 con 0x7f9088102da0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.406+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908819f5b0 con 0x7f9088102da0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.406+0000 7f9085600640 1 -- 172.21.15.103:0/377777726 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f907c0054d0 con 0x7f9088102da0 2024-09-11T00:19:30.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.406+0000 7f9085600640 1 -- 172.21.15.103:0/377777726 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c010690 con 0x7f9088102da0 2024-09-11T00:19:30.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.406+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9088075380 con 0x7f9088102da0 2024-09-11T00:19:30.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.407+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9088103180 con 0x7f9088102da0 2024-09-11T00:19:30.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.409+0000 7f9085600640 1 -- 172.21.15.103:0/377777726 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f907c004e90 con 0x7f9088102da0 2024-09-11T00:19:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.411+0000 7f9085600640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90700b52e0 0x7f90700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:30.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.411+0000 7f9085600640 1 -- 172.21.15.103:0/377777726 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f907c0e5c30 con 0x7f9088102da0 2024-09-11T00:19:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.411+0000 7f9087e00640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90700b52e0 0x7f90700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:30.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.411+0000 7f9087e00640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90700b52e0 0x7f90700b77a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9088108220 tx=0x7f907400f7a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:30.413 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.412+0000 7f9085600640 1 -- 172.21.15.103:0/377777726 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f907c0acef0 con 0x7f9088102da0 2024-09-11T00:19:30.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.597+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90880756a0 con 0x7f9088102da0 2024-09-11T00:19:30.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.598+0000 7f9085600640 1 -- 172.21.15.103:0/377777726 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18500 (secure 0 0 0) 0x7f907c0ac7f0 con 0x7f9088102da0 2024-09-11T00:19:30.600 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:30.600 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":71,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:19:29.856391+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:19:30.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90700b52e0 msgr2=0x7f90700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90700b52e0 0x7f90700b77a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9088108220 tx=0x7f907400f7a0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9088102da0 msgr2=0x7f9088199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9088102da0 0x7f9088199210 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f907c00e9f0 tx=0x7f907c00edf0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 shutdown_connections 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90700b52e0 0x7f90700b77a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9088109130 0x7f908819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90881087b0 0x7f9088199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 --2- 172.21.15.103:0/377777726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9088102da0 0x7f9088199210 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.601+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 >> 172.21.15.103:0/377777726 conn(0x7f90880fe810 msgr2=0x7f9088101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.602+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 shutdown_connections 2024-09-11T00:19:30.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:30.602+0000 7f908f400640 1 -- 172.21.15.103:0/377777726 wait complete. 2024-09-11T00:19:31.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:30 smithi103 ceph-mon[30578]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:19:31.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:30 smithi103 ceph-mon[30578]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:31.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:30 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/377777726' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:19:31.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:30 smithi117 ceph-mon[35525]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:19:31.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:30 smithi117 ceph-mon[35525]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:31.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:30 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/377777726' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:19:31.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:30 smithi183 ceph-mon[35934]: osdmap e71: 12 total, 12 up, 12 in 2024-09-11T00:19:31.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:30 smithi183 ceph-mon[35934]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:31.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:30 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/377777726' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:19:31.522 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T00:15:22.579961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T00:19:31.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T00:19:31.889 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:32.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 -- 172.21.15.103:0/2853086553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 msgr2=0x7fb8301097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 --2- 172.21.15.103:0/2853086553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb8301097b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb820009ca0 tx=0x7fb82002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 -- 172.21.15.103:0/2853086553 shutdown_connections 2024-09-11T00:19:32.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 --2- 172.21.15.103:0/2853086553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb83010f760 0x7fb830115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 --2- 172.21.15.103:0/2853086553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb83010ede0 0x7fb83010f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 --2- 172.21.15.103:0/2853086553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb8301097b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 -- 172.21.15.103:0/2853086553 >> 172.21.15.103:0/2853086553 conn(0x7fb830078a40 msgr2=0x7fb83007ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.484+0000 7fb835400640 1 -- 172.21.15.103:0/2853086553 shutdown_connections 2024-09-11T00:19:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.485+0000 7fb835400640 1 -- 172.21.15.103:0/2853086553 wait complete. 2024-09-11T00:19:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 Processor -- start 2024-09-11T00:19:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 -- start start 2024-09-11T00:19:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb83019f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb83010ede0 0x7fb83019fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb83010f760 0x7fb8301a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8301193e0 con 0x7fb8301093d0 2024-09-11T00:19:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb830119260 con 0x7fb83010f760 2024-09-11T00:19:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.486+0000 7fb835400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb830119560 con 0x7fb83010ede0 2024-09-11T00:19:32.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb83019f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb83019f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33590/0 (socket says 172.21.15.103:33590) 2024-09-11T00:19:32.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82e800640 1 -- 172.21.15.103:0/1232287376 learned_addr learned my addr 172.21.15.103:0/1232287376 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:32.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82f200640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb83010f760 0x7fb8301a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82de00640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb83010ede0 0x7fb83019fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82f200640 1 -- 172.21.15.103:0/1232287376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb83010ede0 msgr2=0x7fb83019fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82f200640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb83010ede0 0x7fb83019fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82f200640 1 -- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 msgr2=0x7fb83019f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82f200640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb83019f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82f200640 1 -- 172.21.15.103:0/1232287376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8301a5a00 con 0x7fb83010f760 2024-09-11T00:19:32.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82e800640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb83019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:32.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.487+0000 7fb82f200640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb83010f760 0x7fb8301a5230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb82800d6b0 tx=0x7fb82800db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.488+0000 7fb827400640 1 -- 172.21.15.103:0/1232287376 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828017070 con 0x7fb83010f760 2024-09-11T00:19:32.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.488+0000 7fb827400640 1 -- 172.21.15.103:0/1232287376 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb828009cd0 con 0x7fb83010f760 2024-09-11T00:19:32.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.488+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8301a5cf0 con 0x7fb83010f760 2024-09-11T00:19:32.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.488+0000 7fb827400640 1 -- 172.21.15.103:0/1232287376 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828003cc0 con 0x7fb83010f760 2024-09-11T00:19:32.495 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.488+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8301ac3b0 con 0x7fb83010f760 2024-09-11T00:19:32.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.489+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8301097b0 con 0x7fb83010f760 2024-09-11T00:19:32.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.491+0000 7fb827400640 1 -- 172.21.15.103:0/1232287376 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fb828009820 con 0x7fb83010f760 2024-09-11T00:19:32.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.492+0000 7fb827400640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8140b5360 0x7fb8140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:32.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.493+0000 7fb827400640 1 -- 172.21.15.103:0/1232287376 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb8280e4ef0 con 0x7fb83010f760 2024-09-11T00:19:32.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.493+0000 7fb82e800640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8140b5360 0x7fb8140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:32.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.493+0000 7fb82e800640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8140b5360 0x7fb8140b7820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb820000f80 tx=0x7fb820005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:32.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.496+0000 7fb827400640 1 -- 172.21.15.103:0/1232287376 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8280abb60 con 0x7fb83010f760 2024-09-11T00:19:32.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:32 smithi103 ceph-mon[30578]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:32.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.681+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb830110fb0 con 0x7fb83010f760 2024-09-11T00:19:32.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.682+0000 7fb827400640 1 -- 172.21.15.103:0/1232287376 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fb8280b0590 con 0x7fb83010f760 2024-09-11T00:19:32.683 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-11T00:19:32.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8140b5360 msgr2=0x7fb8140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8140b5360 0x7fb8140b7820 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb820000f80 tx=0x7fb820005d80 comp rx=0 tx=0).stop 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb83010f760 msgr2=0x7fb8301a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb83010f760 0x7fb8301a5230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb82800d6b0 tx=0x7fb82800db70 comp rx=0 tx=0).stop 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 shutdown_connections 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8140b5360 0x7fb8140b7820 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb83010f760 0x7fb8301a5230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb83010ede0 0x7fb83019fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 --2- 172.21.15.103:0/1232287376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8301093d0 0x7fb83019f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.684+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 >> 172.21.15.103:0/1232287376 conn(0x7fb830078a40 msgr2=0x7fb83007ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.685+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 shutdown_connections 2024-09-11T00:19:32.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:32.685+0000 7fb835400640 1 -- 172.21.15.103:0/1232287376 wait complete. 2024-09-11T00:19:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:32 smithi183 ceph-mon[35934]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:32.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:32 smithi117 ceph-mon[35525]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.a on smithi103 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.d on smithi103 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.g on smithi103 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.j on smithi103 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.b on smithi117 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.e on smithi117 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.h on smithi117 2024-09-11T00:19:33.508 INFO:tasks.cephadm:Adding mds.k on smithi117 2024-09-11T00:19:33.509 INFO:tasks.cephadm:Adding mds.c on smithi183 2024-09-11T00:19:33.509 INFO:tasks.cephadm:Adding mds.f on smithi183 2024-09-11T00:19:33.509 INFO:tasks.cephadm:Adding mds.i on smithi183 2024-09-11T00:19:33.509 INFO:tasks.cephadm:Adding mds.l on smithi183 2024-09-11T00:19:33.509 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l' 2024-09-11T00:19:33.770 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:19:33.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:33 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1232287376' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:19:33.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:33 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1232287376' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:19:33.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:33 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1232287376' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:19:34.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.261+0000 7f964cdf8640 1 -- 172.21.15.183:0/731560023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648108740 msgr2=0x7f964810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:34.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.261+0000 7f964cdf8640 1 --2- 172.21.15.183:0/731560023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648108740 0x7f964810ac00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9640009be0 tx=0x7f964002a6c0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.261+0000 7f964cdf8640 1 -- 172.21.15.183:0/731560023 shutdown_connections 2024-09-11T00:19:34.266 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.261+0000 7f964cdf8640 1 --2- 172.21.15.183:0/731560023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648108740 0x7f964810ac00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.261+0000 7f964cdf8640 1 --2- 172.21.15.183:0/731560023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9648105b90 0x7f9648108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 --2- 172.21.15.183:0/731560023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 0x7f9648105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 -- 172.21.15.183:0/731560023 >> 172.21.15.183:0/731560023 conn(0x7f96480fd250 msgr2=0x7f96480ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 -- 172.21.15.183:0/731560023 shutdown_connections 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 -- 172.21.15.183:0/731560023 wait complete. 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 Processor -- start 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 -- start start 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 0x7f9648198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648105b90 0x7f9648199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9648108740 0x7f9648199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.262+0000 7f964cdf8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f964810e690 con 0x7f9648108740 2024-09-11T00:19:34.267 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964cdf8640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f964810e510 con 0x7f9648105b90 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964cdf8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f964810e810 con 0x7f964806c0e0 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 0x7f9648198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 0x7f9648198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:43458/0 (socket says 172.21.15.183:43458) 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 -- 172.21.15.183:0/2374889951 learned_addr learned my addr 172.21.15.183:0/2374889951 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f963ffff640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648105b90 0x7f9648199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f9646d9d640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9648108740 0x7f9648199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 -- 172.21.15.183:0/2374889951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648105b90 msgr2=0x7f9648199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648105b90 0x7f9648199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 -- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9648108740 msgr2=0x7f9648199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9648108740 0x7f9648199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.268 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 -- 172.21.15.183:0/2374889951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964819a880 con 0x7f964806c0e0 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f9646d9d640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9648108740 0x7f9648199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964659c640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 0x7f9648198dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9c0 tx=0x7f963800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f963f7fe640 1 -- 172.21.15.183:0/2374889951 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963800ccc0 con 0x7f964806c0e0 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964819ab70 con 0x7f964806c0e0 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f963f7fe640 1 -- 172.21.15.183:0/2374889951 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96380054d0 con 0x7f964806c0e0 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f963f7fe640 1 -- 172.21.15.183:0/2374889951 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638010430 con 0x7f964806c0e0 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.263+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96481a1250 con 0x7f964806c0e0 2024-09-11T00:19:34.269 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.264+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9648105ff0 con 0x7f964806c0e0 2024-09-11T00:19:34.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.266+0000 7f963f7fe640 1 -- 172.21.15.183:0/2374889951 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f9638004f80 con 0x7f964806c0e0 2024-09-11T00:19:34.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.267+0000 7f963f7fe640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f96280b52e0 0x7f96280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:34.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.267+0000 7f963f7fe640 1 -- 172.21.15.183:0/2374889951 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f96380e6200 con 0x7f964806c0e0 2024-09-11T00:19:34.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.267+0000 7f963ffff640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f96280b52e0 0x7f96280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:34.270 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.267+0000 7f963ffff640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f96280b52e0 0x7f96280b77a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f964806c590 tx=0x7f9630005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:34.271 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.270+0000 7f963f7fe640 1 -- 172.21.15.183:0/2374889951 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f96380ad560 con 0x7f964806c0e0 2024-09-11T00:19:34.466 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.465+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}) -- 0x7f9648063790 con 0x7f96280b52e0 2024-09-11T00:19:34.473 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.472+0000 7f963f7fe640 1 -- 172.21.15.183:0/2374889951 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f9648063790 con 0x7f96280b52e0 2024-09-11T00:19:34.473 INFO:teuthology.orchestra.run.smithi183.stdout:Scheduled mds.all update... 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f96280b52e0 msgr2=0x7f96280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f96280b52e0 0x7f96280b77a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f964806c590 tx=0x7f9630005f90 comp rx=0 tx=0).stop 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 msgr2=0x7f9648198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 0x7f9648198dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9c0 tx=0x7f963800ee80 comp rx=0 tx=0).stop 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 shutdown_connections 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f96280b52e0 0x7f96280b77a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9648108740 0x7f9648199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9648105b90 0x7f9648199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 --2- 172.21.15.183:0/2374889951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f964806c0e0 0x7f9648198dc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 >> 172.21.15.183:0/2374889951 conn(0x7f96480fd250 msgr2=0x7f9648103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 shutdown_connections 2024-09-11T00:19:34.476 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-11T00:19:34.474+0000 7f964cdf8640 1 -- 172.21.15.183:0/2374889951 wait complete. 2024-09-11T00:19:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:34 smithi183 ceph-mon[35934]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:34.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:34 smithi103 ceph-mon[30578]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:34 smithi117 ceph-mon[35525]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:35.179 DEBUG:teuthology.orchestra.run.smithi103:mds.a> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T00:19:35.183 DEBUG:teuthology.orchestra.run.smithi103:mds.d> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T00:19:35.228 DEBUG:teuthology.orchestra.run.smithi103:mds.g> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T00:19:35.273 DEBUG:teuthology.orchestra.run.smithi103:mds.j> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T00:19:35.318 DEBUG:teuthology.orchestra.run.smithi117:mds.b> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T00:19:35.321 DEBUG:teuthology.orchestra.run.smithi117:mds.e> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T00:19:35.363 DEBUG:teuthology.orchestra.run.smithi117:mds.h> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T00:19:35.406 DEBUG:teuthology.orchestra.run.smithi117:mds.k> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T00:19:35.449 DEBUG:teuthology.orchestra.run.smithi183:mds.c> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T00:19:35.452 DEBUG:teuthology.orchestra.run.smithi183:mds.f> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T00:19:35.495 DEBUG:teuthology.orchestra.run.smithi183:mds.i> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T00:19:35.538 DEBUG:teuthology.orchestra.run.smithi183:mds.l> sudo journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T00:19:35.581 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T00:19:35.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:19:35.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='client.24559 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:19:35.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-11T00:19:35.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:35.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:35.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:35.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:35.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:35.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:35.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:35.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:35.631 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:35 smithi103 ceph-mon[30578]: Deploying daemon mds.l on smithi183 2024-09-11T00:19:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='client.24559 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:19:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-11T00:19:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:35.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:35.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:35.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:35.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:35.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:35.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:35 smithi117 ceph-mon[35525]: Deploying daemon mds.l on smithi183 2024-09-11T00:19:35.719 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='client.24559 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: Saving service mds.all spec with placement smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:35.720 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:35 smithi183 ceph-mon[35934]: Deploying daemon mds.l on smithi183 2024-09-11T00:19:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:36.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:36 smithi103 ceph-mon[30578]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 -- 172.21.15.103:0/518739806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa010a820 msgr2=0x7fdfa010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 --2- 172.21.15.103:0/518739806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa010a820 0x7fdfa010ac60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdf88009900 tx=0x7fdf8802a410 comp rx=0 tx=0).stop 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 -- 172.21.15.103:0/518739806 shutdown_connections 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 --2- 172.21.15.103:0/518739806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 0x7fdfa01119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 --2- 172.21.15.103:0/518739806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa010a820 0x7fdfa010ac60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 --2- 172.21.15.103:0/518739806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfa0104e10 0x7fdfa01051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 -- 172.21.15.103:0/518739806 >> 172.21.15.103:0/518739806 conn(0x7fdfa0100840 msgr2=0x7fdfa0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 -- 172.21.15.103:0/518739806 shutdown_connections 2024-09-11T00:19:36.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.616+0000 7fdf9f200640 1 -- 172.21.15.103:0/518739806 wait complete. 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.617+0000 7fdf9f200640 1 Processor -- start 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9f200640 1 -- start start 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa0104e10 0x7fdfa019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfa010a820 0x7fdfa019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 0x7fdfa01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0114eb0 con 0x7fdfa010b1a0 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0114d30 con 0x7fdfa0104e10 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa0115030 con 0x7fdfa010a820 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 0x7fdfa01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf97e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfa010a820 0x7fdfa019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 0x7fdfa01a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33614/0 (socket says 172.21.15.103:33614) 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9e800640 1 -- 172.21.15.103:0/3681618268 learned_addr learned my addr 172.21.15.103:0/3681618268 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:36.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.618+0000 7fdf9de00640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa0104e10 0x7fdfa019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9e800640 1 -- 172.21.15.103:0/3681618268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfa010a820 msgr2=0x7fdfa019b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9e800640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfa010a820 0x7fdfa019b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9e800640 1 -- 172.21.15.103:0/3681618268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa0104e10 msgr2=0x7fdfa019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9e800640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa0104e10 0x7fdfa019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9e800640 1 -- 172.21.15.103:0/3681618268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa01a13b0 con 0x7fdfa010b1a0 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf97e00640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfa010a820 0x7fdfa019b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9de00640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa0104e10 0x7fdfa019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9e800640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 0x7fdfa01a0be0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800eb10 tx=0x7fdf9800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf96a00640 1 -- 172.21.15.103:0/3681618268 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98004b20 con 0x7fdfa010b1a0 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf96a00640 1 -- 172.21.15.103:0/3681618268 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf98004cc0 con 0x7fdfa010b1a0 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa01a16a0 con 0x7fdfa010b1a0 2024-09-11T00:19:36.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.619+0000 7fdf96a00640 1 -- 172.21.15.103:0/3681618268 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98010670 con 0x7fdfa010b1a0 2024-09-11T00:19:36.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.620+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa01a7d60 con 0x7fdfa010b1a0 2024-09-11T00:19:36.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.621+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa01051f0 con 0x7fdfa010b1a0 2024-09-11T00:19:36.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.624+0000 7fdf96a00640 1 -- 172.21.15.103:0/3681618268 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fdf98005020 con 0x7fdfa010b1a0 2024-09-11T00:19:36.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.625+0000 7fdf96a00640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdf840b5410 0x7fdf840b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:36.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.625+0000 7fdf96a00640 1 -- 172.21.15.103:0/3681618268 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fdf980e5c80 con 0x7fdfa010b1a0 2024-09-11T00:19:36.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.625+0000 7fdf9de00640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdf840b5410 0x7fdf840b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:36.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.626+0000 7fdf9de00640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdf840b5410 0x7fdf840b78d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009fc0 tx=0x7fdf90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:36.629 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:36 smithi183 ceph-mon[35934]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:36.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.629+0000 7fdf96a00640 1 -- 172.21.15.103:0/3681618268 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdf980acf40 con 0x7fdfa010b1a0 2024-09-11T00:19:36.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.847+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfa01a80c0 con 0x7fdfa010b1a0 2024-09-11T00:19:36.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.848+0000 7fdf96a00640 1 -- 172.21.15.103:0/3681618268 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fdf980ac840 con 0x7fdfa010b1a0 2024-09-11T00:19:36.850 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:36.850 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":1,"btime":"2024-09-11T00:11:30:997230+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T00:19:36.850 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 1 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdf840b5410 msgr2=0x7fdf840b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdf840b5410 0x7fdf840b78d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009fc0 tx=0x7fdf90005f90 comp rx=0 tx=0).stop 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 msgr2=0x7fdfa01a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 0x7fdfa01a0be0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800eb10 tx=0x7fdf9800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 shutdown_connections 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdf840b5410 0x7fdf840b78d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdfa010b1a0 0x7fdfa01a0be0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfa010a820 0x7fdfa019b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 --2- 172.21.15.103:0/3681618268 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfa0104e10 0x7fdfa019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:36.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 >> 172.21.15.103:0/3681618268 conn(0x7fdfa0100840 msgr2=0x7fdfa0103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:36.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 shutdown_connections 2024-09-11T00:19:36.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:36.850+0000 7fdf9f200640 1 -- 172.21.15.103:0/3681618268 wait complete. 2024-09-11T00:19:36.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:36 smithi117 ceph-mon[35525]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:37.280 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 systemd[1]: Starting Ceph mds.l for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:37.560 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:37 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3681618268' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:37.560 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 podman[65134]: 2024-09-11 00:19:37.19254623 +0000 UTC m=+0.015522433 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:19:37.560 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 podman[65134]: 2024-09-11 00:19:37.314219133 +0000 UTC m=+0.137195326 container create 82cb105a7b6ffa58e9ef6831a29378a28b7ef8535e951918e77ae800256bb27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T00:19:37.560 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 podman[65134]: 2024-09-11 00:19:37.558907584 +0000 UTC m=+0.381883777 container init 82cb105a7b6ffa58e9ef6831a29378a28b7ef8535e951918e77ae800256bb27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:19:37.620 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:37 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3681618268' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:37.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:37 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3681618268' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:38.030 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 podman[65134]: 2024-09-11 00:19:37.56217702 +0000 UTC m=+0.385153226 container start 82cb105a7b6ffa58e9ef6831a29378a28b7ef8535e951918e77ae800256bb27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T00:19:38.031 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 ceph-mds[65153]: -- 172.21.15.183:0/1833420847 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbe1117860 con 0x55cbe10f4800 2024-09-11T00:19:38.031 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l[65149]: starting mds.l at 2024-09-11T00:19:38.031 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 bash[65134]: 82cb105a7b6ffa58e9ef6831a29378a28b7ef8535e951918e77ae800256bb27d 2024-09-11T00:19:38.031 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:19:37 smithi183 systemd[1]: Started Ceph mds.l for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:19:38.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6832/1084194007,v1:172.21.15.183:6833/1084194007] up:boot 2024-09-11T00:19:38.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: fsmap 1 up:standby 2024-09-11T00:19:38.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:19:38.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:38.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:38.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:38.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:38.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:38.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:38.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: Deploying daemon mds.k on smithi117 2024-09-11T00:19:38.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:38 smithi103 ceph-mon[30578]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6832/1084194007,v1:172.21.15.183:6833/1084194007] up:boot 2024-09-11T00:19:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: fsmap 1 up:standby 2024-09-11T00:19:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:19:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: Deploying daemon mds.k on smithi117 2024-09-11T00:19:38.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:38 smithi117 ceph-mon[35525]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:39.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6832/1084194007,v1:172.21.15.183:6833/1084194007] up:boot 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: fsmap 1 up:standby 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: Deploying daemon mds.k on smithi117 2024-09-11T00:19:39.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:38 smithi183 ceph-mon[35934]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:39.662 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:19:39.937 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:39 smithi117 systemd[1]: Starting Ceph mds.k for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:40.037 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:40.338 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 podman[64351]: 2024-09-11 00:19:39.949579381 +0000 UTC m=+0.015879056 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:19:40.338 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 podman[64351]: 2024-09-11 00:19:40.067635277 +0000 UTC m=+0.133934940 container create f4c80cfd8ddd2bcc098143e404e8106866da52e3dd7e4b07a036e0bbf11764ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T00:19:40.338 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 podman[64351]: 2024-09-11 00:19:40.337082379 +0000 UTC m=+0.403382044 container init f4c80cfd8ddd2bcc098143e404e8106866da52e3dd7e4b07a036e0bbf11764ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2) 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 -- 172.21.15.103:0/907759433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075f40 msgr2=0x7f1c4010ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 --2- 172.21.15.103:0/907759433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075f40 0x7f1c4010ec50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009860 tx=0x7f1c2802a340 comp rx=0 tx=0).stop 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 -- 172.21.15.103:0/907759433 shutdown_connections 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 --2- 172.21.15.103:0/907759433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4010f220 0x7f1c40111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 --2- 172.21.15.103:0/907759433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075f40 0x7f1c4010ec50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 --2- 172.21.15.103:0/907759433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c40075540 0x7f1c40075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 -- 172.21.15.103:0/907759433 >> 172.21.15.103:0/907759433 conn(0x7f1c400fe6d0 msgr2=0x7f1c40100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 -- 172.21.15.103:0/907759433 shutdown_connections 2024-09-11T00:19:40.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.639+0000 7f1c44a00640 1 -- 172.21.15.103:0/907759433 wait complete. 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.640+0000 7f1c44a00640 1 Processor -- start 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.640+0000 7f1c44a00640 1 -- start start 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.640+0000 7f1c44a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075540 0x7f1c4019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.640+0000 7f1c44a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c40075f40 0x7f1c4019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.640+0000 7f1c44a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4010f220 0x7f1c401a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c44a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c40114dd0 con 0x7f1c4010f220 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c44a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1c40114c50 con 0x7f1c40075540 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c44a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1c40114f50 con 0x7f1c40075f40 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075540 0x7f1c4019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075540 0x7f1c4019b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:39146/0 (socket says 172.21.15.103:39146) 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 -- 172.21.15.103:0/2869757645 learned_addr learned my addr 172.21.15.103:0/2869757645 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c37e00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c40075f40 0x7f1c4019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3fe00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4010f220 0x7f1c401a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:40.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 -- 172.21.15.103:0/2869757645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c40075f40 msgr2=0x7f1c4019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c40075f40 0x7f1c4019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 -- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4010f220 msgr2=0x7f1c401a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4010f220 0x7f1c401a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 -- 172.21.15.103:0/2869757645 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c401a12f0 con 0x7f1c40075540 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3fe00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4010f220 0x7f1c401a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.641+0000 7f1c3f400640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075540 0x7f1c4019b240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000e9f0 tx=0x7f1c3000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.642+0000 7f1c3cc00640 1 -- 172.21.15.103:0/2869757645 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3000cd80 con 0x7f1c40075540 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.642+0000 7f1c3cc00640 1 -- 172.21.15.103:0/2869757645 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c300108e0 con 0x7f1c40075540 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.642+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c401a15e0 con 0x7f1c40075540 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.642+0000 7f1c3cc00640 1 -- 172.21.15.103:0/2869757645 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30018980 con 0x7f1c40075540 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.642+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c401a7e70 con 0x7f1c40075540 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.643+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c40075920 con 0x7f1c40075540 2024-09-11T00:19:40.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.645+0000 7f1c3cc00640 1 -- 172.21.15.103:0/2869757645 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f1c30010430 con 0x7f1c40075540 2024-09-11T00:19:40.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.647+0000 7f1c3cc00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1c240b5490 0x7f1c240b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:40.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.647+0000 7f1c3cc00640 1 -- 172.21.15.103:0/2869757645 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f1c300e5b50 con 0x7f1c40075540 2024-09-11T00:19:40.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.647+0000 7f1c37e00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1c240b5490 0x7f1c240b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:40.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.648+0000 7f1c37e00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1c240b5490 0x7f1c240b7950 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009830 tx=0x7f1c28030340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:40.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.650+0000 7f1c3cc00640 1 -- 172.21.15.103:0/2869757645 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1c300ace10 con 0x7f1c40075540 2024-09-11T00:19:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:40 smithi117 ceph-mon[35525]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:40 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6832/928029640,v1:172.21.15.117:6833/928029640] up:boot 2024-09-11T00:19:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:40 smithi117 ceph-mon[35525]: fsmap 2 up:standby 2024-09-11T00:19:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:19:40.687 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 podman[64351]: 2024-09-11 00:19:40.340061588 +0000 UTC m=+0.406361251 container start f4c80cfd8ddd2bcc098143e404e8106866da52e3dd7e4b07a036e0bbf11764ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-11T00:19:40.687 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 ceph-mds[64370]: -- 172.21.15.117:0/3043884887 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56119d949860 con 0x56119d926800 2024-09-11T00:19:40.687 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k[64366]: starting mds.k at 2024-09-11T00:19:40.688 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 bash[64351]: f4c80cfd8ddd2bcc098143e404e8106866da52e3dd7e4b07a036e0bbf11764ef 2024-09-11T00:19:40.688 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:19:40 smithi117 systemd[1]: Started Ceph mds.k for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:19:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:40 smithi183 ceph-mon[35934]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:40 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6832/928029640,v1:172.21.15.117:6833/928029640] up:boot 2024-09-11T00:19:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:40 smithi183 ceph-mon[35934]: fsmap 2 up:standby 2024-09-11T00:19:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:19:40.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.881+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c4019c370 con 0x7f1c40075540 2024-09-11T00:19:40.884 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:40 smithi103 ceph-mon[30578]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:40.884 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:40 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6832/928029640,v1:172.21.15.117:6833/928029640] up:boot 2024-09-11T00:19:40.884 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:40 smithi103 ceph-mon[30578]: fsmap 2 up:standby 2024-09-11T00:19:40.884 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-11T00:19:40.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.882+0000 7f1c3cc00640 1 -- 172.21.15.103:0/2869757645 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f1c300ac710 con 0x7f1c40075540 2024-09-11T00:19:40.884 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:40.885 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":3,"btime":"2024-09-11T00:19:40:418258+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T00:19:40.885 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 3 2024-09-11T00:19:40.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1c240b5490 msgr2=0x7f1c240b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:40.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1c240b5490 0x7f1c240b7950 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c28009830 tx=0x7f1c28030340 comp rx=0 tx=0).stop 2024-09-11T00:19:40.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075540 msgr2=0x7f1c4019b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:40.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075540 0x7f1c4019b240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000e9f0 tx=0x7f1c3000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 shutdown_connections 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f1c240b5490 0x7f1c240b7950 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c4010f220 0x7f1c401a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1c40075f40 0x7f1c4019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 --2- 172.21.15.103:0/2869757645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c40075540 0x7f1c4019b240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 >> 172.21.15.103:0/2869757645 conn(0x7f1c400fe6d0 msgr2=0x7f1c400ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.885+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 shutdown_connections 2024-09-11T00:19:40.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:40.886+0000 7f1c44a00640 1 -- 172.21.15.103:0/2869757645 wait complete. 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: Deploying daemon mds.j on smithi103 2024-09-11T00:19:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:41 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2869757645' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:41.764 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.765 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.765 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.765 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:41.765 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:41.765 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:41.765 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: Deploying daemon mds.j on smithi103 2024-09-11T00:19:41.765 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:41 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2869757645' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: Deploying daemon mds.j on smithi103 2024-09-11T00:19:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:41 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2869757645' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:42 smithi183 ceph-mon[35934]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:42.910 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:42 smithi103 ceph-mon[30578]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:42.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:42 smithi117 ceph-mon[35525]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:43.854 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:19:44.384 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 systemd[1]: Starting Ceph mds.j for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:44 smithi103 ceph-mon[30578]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:44.670 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 podman[71675]: 2024-09-11 00:19:44.288627948 +0000 UTC m=+0.032615337 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:19:44.670 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 podman[71675]: 2024-09-11 00:19:44.474809419 +0000 UTC m=+0.218796790 container create 59160ec8d9488713897aa367834ef0749ef8e1756db8009341d897982d461d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T00:19:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:44 smithi183 ceph-mon[35934]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:44 smithi117 ceph-mon[35525]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:45.014 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:45.154 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 podman[71675]: 2024-09-11 00:19:44.730663475 +0000 UTC m=+0.474650847 container init 59160ec8d9488713897aa367834ef0749ef8e1756db8009341d897982d461d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T00:19:45.154 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 podman[71675]: 2024-09-11 00:19:44.737499376 +0000 UTC m=+0.481486757 container start 59160ec8d9488713897aa367834ef0749ef8e1756db8009341d897982d461d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:19:45.154 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 ceph-mds[71710]: -- 172.21.15.103:0/145281231 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2615ab860 con 0x55d261588800 2024-09-11T00:19:45.154 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j[71706]: starting mds.j at 2024-09-11T00:19:45.154 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 bash[71675]: 59160ec8d9488713897aa367834ef0749ef8e1756db8009341d897982d461d32 2024-09-11T00:19:45.154 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:19:44 smithi103 systemd[1]: Started Ceph mds.j for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 -- 172.21.15.103:0/7045476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 msgr2=0x7f4a141053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 --2- 172.21.15.103:0/7045476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a141053a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009900 tx=0x7f4a0802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 -- 172.21.15.103:0/7045476 shutdown_connections 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 --2- 172.21.15.103:0/7045476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a1410b350 0x7f4a14111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 --2- 172.21.15.103:0/7045476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1410a9d0 0x7f4a1410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 --2- 172.21.15.103:0/7045476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a141053a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 -- 172.21.15.103:0/7045476 >> 172.21.15.103:0/7045476 conn(0x7f4a141009f0 msgr2=0x7f4a14102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 -- 172.21.15.103:0/7045476 shutdown_connections 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.630+0000 7f4a1c200640 1 -- 172.21.15.103:0/7045476 wait complete. 2024-09-11T00:19:45.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.631+0000 7f4a1c200640 1 Processor -- start 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.631+0000 7f4a1c200640 1 -- start start 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1c200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a1419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1c200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a1410a9d0 0x7f4a1419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1c200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1410b350 0x7f4a141a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1c200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14114fb0 con 0x7f4a14104fc0 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1c200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14114e30 con 0x7f4a1410b350 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1c200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14115130 con 0x7f4a1410a9d0 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a1419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a1419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50946/0 (socket says 172.21.15.103:50946) 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 -- 172.21.15.103:0/3450886295 learned_addr learned my addr 172.21.15.103:0/3450886295 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a18e00640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a1410a9d0 0x7f4a1419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1a200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1410b350 0x7f4a141a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 -- 172.21.15.103:0/3450886295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a1410a9d0 msgr2=0x7f4a1419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a1410a9d0 0x7f4a1419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 -- 172.21.15.103:0/3450886295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1410b350 msgr2=0x7f4a141a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1410b350 0x7f4a141a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a19800640 1 -- 172.21.15.103:0/3450886295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a141a1530 con 0x7f4a14104fc0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.632+0000 7f4a1a200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1410b350 0x7f4a141a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.633+0000 7f4a19800640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a1419b480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005ab0 tx=0x7f4a08003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.633+0000 7f4a12000640 1 -- 172.21.15.103:0/3450886295 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0803f070 con 0x7f4a14104fc0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.633+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a141a17c0 con 0x7f4a14104fc0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.633+0000 7f4a12000640 1 -- 172.21.15.103:0/3450886295 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a08031d60 con 0x7f4a14104fc0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.633+0000 7f4a12000640 1 -- 172.21.15.103:0/3450886295 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08035030 con 0x7f4a14104fc0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.633+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a141a7ee0 con 0x7f4a14104fc0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.634+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a141053a0 con 0x7f4a14104fc0 2024-09-11T00:19:45.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.636+0000 7f4a12000640 1 -- 172.21.15.103:0/3450886295 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f4a0803b560 con 0x7f4a14104fc0 2024-09-11T00:19:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.637+0000 7f4a12000640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49fc0b5360 0x7f49fc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.638+0000 7f4a12000640 1 -- 172.21.15.103:0/3450886295 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f4a08106310 con 0x7f4a14104fc0 2024-09-11T00:19:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.638+0000 7f4a18e00640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49fc0b5360 0x7f49fc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.638+0000 7f4a18e00640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49fc0b5360 0x7f49fc0b7820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a1410a550 tx=0x7f4a0000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:45.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.642+0000 7f4a12000640 1 -- 172.21.15.103:0/3450886295 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4a080cd5d0 con 0x7f4a14104fc0 2024-09-11T00:19:45.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.892+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a141a8250 con 0x7f4a14104fc0 2024-09-11T00:19:45.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.892+0000 7f4a12000640 1 -- 172.21.15.103:0/3450886295 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f4a080cced0 con 0x7f4a14104fc0 2024-09-11T00:19:45.894 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:45.894 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":4,"btime":"2024-09-11T00:19:45:444332+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T00:19:45.895 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 4 2024-09-11T00:19:45.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49fc0b5360 msgr2=0x7f49fc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49fc0b5360 0x7f49fc0b7820 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4a1410a550 tx=0x7f4a0000b390 comp rx=0 tx=0).stop 2024-09-11T00:19:45.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 msgr2=0x7f4a1419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:45.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a1419b480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005ab0 tx=0x7f4a08003cd0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 shutdown_connections 2024-09-11T00:19:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49fc0b5360 0x7f49fc0b7820 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a1410b350 0x7f4a141a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a1410a9d0 0x7f4a1419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 --2- 172.21.15.103:0/3450886295 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a14104fc0 0x7f4a1419b480 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 >> 172.21.15.103:0/3450886295 conn(0x7f4a141009f0 msgr2=0x7f4a14101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 shutdown_connections 2024-09-11T00:19:45.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:45.895+0000 7f4a1c200640 1 -- 172.21.15.103:0/3450886295 wait complete. 2024-09-11T00:19:46.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: Deploying daemon mds.d on smithi103 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.103:6834/1632166659,v1:172.21.15.103:6835/1632166659] up:boot 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: fsmap 3 up:standby 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:19:46.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:45 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3450886295' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:46.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: Deploying daemon mds.d on smithi103 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.103:6834/1632166659,v1:172.21.15.103:6835/1632166659] up:boot 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: fsmap 3 up:standby 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:19:46.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:45 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3450886295' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:46.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:46.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:46.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:46.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:46.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: Deploying daemon mds.d on smithi103 2024-09-11T00:19:46.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.103:6834/1632166659,v1:172.21.15.103:6835/1632166659] up:boot 2024-09-11T00:19:46.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: fsmap 3 up:standby 2024-09-11T00:19:46.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-11T00:19:46.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:45 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3450886295' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:47.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:46 smithi183 ceph-mon[35934]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:47.361 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:46 smithi103 ceph-mon[30578]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:47.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:46 smithi117 ceph-mon[35525]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:48.642 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:48 smithi103 ceph-mon[30578]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:48 smithi183 ceph-mon[35934]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:48.920 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:48 smithi103 systemd[1]: Starting Ceph mds.d for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:48 smithi117 ceph-mon[35525]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:49.263 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:48 smithi103 podman[72059]: 2024-09-11 00:19:48.857006783 +0000 UTC m=+0.027118408 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:19:49.263 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:49 smithi103 podman[72059]: 2024-09-11 00:19:49.033936507 +0000 UTC m=+0.204048122 container create da856ad467f40cb5a5af0befdee1cf76470fbb19d4331bcc1aebbec160e455a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T00:19:49.263 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:49 smithi103 podman[72059]: 2024-09-11 00:19:49.261817682 +0000 UTC m=+0.431929319 container init da856ad467f40cb5a5af0befdee1cf76470fbb19d4331bcc1aebbec160e455a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T00:19:49.401 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:19:49.670 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:49 smithi103 podman[72059]: 2024-09-11 00:19:49.268034935 +0000 UTC m=+0.438146551 container start da856ad467f40cb5a5af0befdee1cf76470fbb19d4331bcc1aebbec160e455a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T00:19:49.670 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:49 smithi103 ceph-mds[72078]: -- 172.21.15.103:0/2518246024 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f3cdc5860 con 0x561f3cda3000 2024-09-11T00:19:49.670 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d[72074]: starting mds.d at 2024-09-11T00:19:49.670 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:49 smithi103 bash[72059]: da856ad467f40cb5a5af0befdee1cf76470fbb19d4331bcc1aebbec160e455a3 2024-09-11T00:19:49.670 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:19:49 smithi103 systemd[1]: Started Ceph mds.d for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:19:49.791 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:50.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.410+0000 7f5aeb200640 1 -- 172.21.15.103:0/94611356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae40776d0 msgr2=0x7f5ae4077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.410+0000 7f5aeb200640 1 --2- 172.21.15.103:0/94611356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae40776d0 0x7f5ae4077ab0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009900 tx=0x7f5ad802a410 comp rx=0 tx=0).stop 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.410+0000 7f5aeb200640 1 -- 172.21.15.103:0/94611356 shutdown_connections 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.410+0000 7f5aeb200640 1 --2- 172.21.15.103:0/94611356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae4113800 0x7f5ae4115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.410+0000 7f5aeb200640 1 --2- 172.21.15.103:0/94611356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae4077ff0 0x7f5ae41132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.410+0000 7f5aeb200640 1 --2- 172.21.15.103:0/94611356 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae40776d0 0x7f5ae4077ab0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.410+0000 7f5aeb200640 1 -- 172.21.15.103:0/94611356 >> 172.21.15.103:0/94611356 conn(0x7f5ae4100840 msgr2=0x7f5ae4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.411+0000 7f5aeb200640 1 -- 172.21.15.103:0/94611356 shutdown_connections 2024-09-11T00:19:50.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.411+0000 7f5aeb200640 1 -- 172.21.15.103:0/94611356 wait complete. 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.411+0000 7f5aeb200640 1 Processor -- start 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aeb200640 1 -- start start 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aeb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae40776d0 0x7f5ae419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aeb200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae4077ff0 0x7f5ae419fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aeb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae4113800 0x7f5ae41a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aeb200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4119200 con 0x7f5ae4077ff0 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aeb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4119080 con 0x7f5ae4113800 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aeb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae4119380 con 0x7f5ae40776d0 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5ae9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae4077ff0 0x7f5ae419fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5ae9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae40776d0 0x7f5ae419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5ae9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae40776d0 0x7f5ae419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:52124/0 (socket says 172.21.15.103:52124) 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5ae9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae4077ff0 0x7f5ae419fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50984/0 (socket says 172.21.15.103:50984) 2024-09-11T00:19:50.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5ae9e00640 1 -- 172.21.15.103:0/3339797673 learned_addr learned my addr 172.21.15.103:0/3339797673 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.412+0000 7f5aea800640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae4113800 0x7f5ae41a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5ae9e00640 1 -- 172.21.15.103:0/3339797673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae4113800 msgr2=0x7f5ae41a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5ae9e00640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae4113800 0x7f5ae41a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5ae9e00640 1 -- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae4077ff0 msgr2=0x7f5ae419fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5ae9e00640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae4077ff0 0x7f5ae419fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5ae9e00640 1 -- 172.21.15.103:0/3339797673 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae41a5700 con 0x7f5ae40776d0 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5ae9e00640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae40776d0 0x7f5ae419f650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005940 tx=0x7f5ad80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5adea00640 1 -- 172.21.15.103:0/3339797673 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad803f030 con 0x7f5ae40776d0 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5adea00640 1 -- 172.21.15.103:0/3339797673 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad80044c0 con 0x7f5ae40776d0 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5adea00640 1 -- 172.21.15.103:0/3339797673 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8035070 con 0x7f5ae40776d0 2024-09-11T00:19:50.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae41a5990 con 0x7f5ae40776d0 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.413+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae41ac0b0 con 0x7f5ae40776d0 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.414+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae4077ab0 con 0x7f5ae40776d0 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.417+0000 7f5adea00640 1 -- 172.21.15.103:0/3339797673 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f5ad803ba20 con 0x7f5ae40776d0 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.418+0000 7f5adea00640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5acc0b5390 0x7f5acc0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.418+0000 7f5adea00640 1 -- 172.21.15.103:0/3339797673 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5ad8105fe0 con 0x7f5ae40776d0 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.418+0000 7f5ae9400640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5acc0b5390 0x7f5acc0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.419+0000 7f5ae9400640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5acc0b5390 0x7f5acc0b7850 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ae407a080 tx=0x7f5ad4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:50.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.420+0000 7f5adea00640 1 -- 172.21.15.103:0/3339797673 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ad80ccc90 con 0x7f5ae40776d0 2024-09-11T00:19:50.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.645+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ae41ac310 con 0x7f5ae40776d0 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.103:6836/3157272512,v1:172.21.15.103:6837/3157272512] up:boot 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: fsmap 4 up:standby 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:50.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: Deploying daemon mds.e on smithi117 2024-09-11T00:19:50.649 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:50 smithi103 ceph-mon[30578]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:50.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.646+0000 7f5adea00640 1 -- 172.21.15.103:0/3339797673 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f5ad80d16c0 con 0x7f5ae40776d0 2024-09-11T00:19:50.649 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:50.649 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":5,"btime":"2024-09-11T00:19:49:444351+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T00:19:50.650 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 5 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.648+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5acc0b5390 msgr2=0x7f5acc0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.648+0000 7f5aeb200640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5acc0b5390 0x7f5acc0b7850 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ae407a080 tx=0x7f5ad4005f90 comp rx=0 tx=0).stop 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.648+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae40776d0 msgr2=0x7f5ae419f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.648+0000 7f5aeb200640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae40776d0 0x7f5ae419f650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8005940 tx=0x7f5ad80040c0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 shutdown_connections 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5acc0b5390 0x7f5acc0b7850 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ae4113800 0x7f5ae41a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ae4077ff0 0x7f5ae419fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 --2- 172.21.15.103:0/3339797673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ae40776d0 0x7f5ae419f650 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 >> 172.21.15.103:0/3339797673 conn(0x7f5ae4100840 msgr2=0x7f5ae4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:50.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 shutdown_connections 2024-09-11T00:19:50.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:50.649+0000 7f5aeb200640 1 -- 172.21.15.103:0/3339797673 wait complete. 2024-09-11T00:19:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.103:6836/3157272512,v1:172.21.15.103:6837/3157272512] up:boot 2024-09-11T00:19:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: fsmap 4 up:standby 2024-09-11T00:19:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:19:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: Deploying daemon mds.e on smithi117 2024-09-11T00:19:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:50 smithi117 ceph-mon[35525]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.103:6836/3157272512,v1:172.21.15.103:6837/3157272512] up:boot 2024-09-11T00:19:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: fsmap 4 up:standby 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: Deploying daemon mds.e on smithi117 2024-09-11T00:19:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:50 smithi183 ceph-mon[35934]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:51 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3339797673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:51.688 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:51 smithi117 systemd[1]: Starting Ceph mds.e for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:51.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:51 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3339797673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:51.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:51 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3339797673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:52.018 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:51 smithi117 podman[64665]: 2024-09-11 00:19:51.641159343 +0000 UTC m=+0.015616973 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:19:52.018 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:51 smithi117 podman[64665]: 2024-09-11 00:19:51.747119796 +0000 UTC m=+0.121577414 container create 6baff9b7466e6d17d9616fa99cb7b811fc4921696e2f73267525e5f8d834f471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T00:19:52.018 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:52 smithi117 podman[64665]: 2024-09-11 00:19:52.016718443 +0000 UTC m=+0.391176060 container init 6baff9b7466e6d17d9616fa99cb7b811fc4921696e2f73267525e5f8d834f471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:19:52.437 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:52 smithi117 podman[64665]: 2024-09-11 00:19:52.01988407 +0000 UTC m=+0.394341697 container start 6baff9b7466e6d17d9616fa99cb7b811fc4921696e2f73267525e5f8d834f471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T00:19:52.437 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:52 smithi117 ceph-mds[64684]: -- 172.21.15.117:0/3810901540 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8d83a1860 con 0x55e8d837f000 2024-09-11T00:19:52.437 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e[64680]: starting mds.e at 2024-09-11T00:19:52.438 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:52 smithi117 bash[64665]: 6baff9b7466e6d17d9616fa99cb7b811fc4921696e2f73267525e5f8d834f471 2024-09-11T00:19:52.438 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:19:52 smithi117 systemd[1]: Started Ceph mds.e for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:19:53.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:53.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:53.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:53.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: Deploying daemon mds.f on smithi183 2024-09-11T00:19:53.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:53.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6834/2973217634,v1:172.21.15.117:6835/2973217634] up:boot 2024-09-11T00:19:53.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: fsmap 5 up:standby 2024-09-11T00:19:53.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: Deploying daemon mds.f on smithi183 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6834/2973217634,v1:172.21.15.117:6835/2973217634] up:boot 2024-09-11T00:19:53.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: fsmap 5 up:standby 2024-09-11T00:19:53.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:19:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:53.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: Deploying daemon mds.f on smithi183 2024-09-11T00:19:53.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:53.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6834/2973217634,v1:172.21.15.117:6835/2973217634] up:boot 2024-09-11T00:19:53.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: fsmap 5 up:standby 2024-09-11T00:19:53.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:19:53.513 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:19:53.897 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:54.281 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 systemd[1]: Starting Ceph mds.f for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:54.498 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:54 smithi103 ceph-mon[30578]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 -- 172.21.15.103:0/3319128577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 msgr2=0x7feef8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 --2- 172.21.15.103:0/3319128577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 0x7feef8102ff0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feee8009900 tx=0x7feee802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 -- 172.21.15.103:0/3319128577 shutdown_connections 2024-09-11T00:19:54.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 --2- 172.21.15.103:0/3319128577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 0x7feef810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 --2- 172.21.15.103:0/3319128577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feef8108620 0x7feef8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 --2- 172.21.15.103:0/3319128577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 0x7feef8102ff0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 -- 172.21.15.103:0/3319128577 >> 172.21.15.103:0/3319128577 conn(0x7feef80fe6c0 msgr2=0x7feef8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 -- 172.21.15.103:0/3319128577 shutdown_connections 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.495+0000 7feefe200640 1 -- 172.21.15.103:0/3319128577 wait complete. 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.496+0000 7feefe200640 1 Processor -- start 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.496+0000 7feefe200640 1 -- start start 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefe200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 0x7feef819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefe200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feef8108620 0x7feef819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefe200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 0x7feef81a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefe200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feef8112c60 con 0x7feef8108fa0 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefe200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feef8112ae0 con 0x7feef8108620 2024-09-11T00:19:54.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefe200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feef8112de0 con 0x7feef8102c10 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefd800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 0x7feef81a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 0x7feef819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefd800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 0x7feef81a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41214/0 (socket says 172.21.15.103:41214) 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feefd800640 1 -- 172.21.15.103:0/1211204807 learned_addr learned my addr 172.21.15.103:0/1211204807 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.497+0000 7feeefe00640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feef8108620 0x7feef819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefd800640 1 -- 172.21.15.103:0/1211204807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 msgr2=0x7feef819b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefd800640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 0x7feef819b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefd800640 1 -- 172.21.15.103:0/1211204807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feef8108620 msgr2=0x7feef819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefd800640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feef8108620 0x7feef819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefd800640 1 -- 172.21.15.103:0/1211204807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef81a1300 con 0x7feef8108fa0 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefce00640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 0x7feef819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefd800640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 0x7feef81a0b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feef000eb10 tx=0x7feef000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:54.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feef6000640 1 -- 172.21.15.103:0/1211204807 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0004b20 con 0x7feef8108fa0 2024-09-11T00:19:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feef6000640 1 -- 172.21.15.103:0/1211204807 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef0004cc0 con 0x7feef8108fa0 2024-09-11T00:19:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef81a15f0 con 0x7feef8108fa0 2024-09-11T00:19:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feef6000640 1 -- 172.21.15.103:0/1211204807 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0010670 con 0x7feef8108fa0 2024-09-11T00:19:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.498+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef81a7cb0 con 0x7feef8108fa0 2024-09-11T00:19:54.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.499+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feef8102ff0 con 0x7feef8108fa0 2024-09-11T00:19:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.502+0000 7feef6000640 1 -- 172.21.15.103:0/1211204807 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7feef0005000 con 0x7feef8108fa0 2024-09-11T00:19:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.503+0000 7feef6000640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feedc0b5410 0x7feedc0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.503+0000 7feef6000640 1 -- 172.21.15.103:0/1211204807 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7feef00e5f10 con 0x7feef8108fa0 2024-09-11T00:19:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.503+0000 7feefce00640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feedc0b5410 0x7feedc0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:54.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.504+0000 7feefce00640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feedc0b5410 0x7feedc0b78d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feee8031440 tx=0x7feee8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:54.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.505+0000 7feef6000640 1 -- 172.21.15.103:0/1211204807 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7feef0056220 con 0x7feef8108fa0 2024-09-11T00:19:54.655 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:54 smithi183 ceph-mon[35934]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:54.656 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 podman[65436]: 2024-09-11 00:19:54.293677619 +0000 UTC m=+0.015865582 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:19:54.656 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 podman[65436]: 2024-09-11 00:19:54.4188896 +0000 UTC m=+0.141077556 container create c8584de6ddb13ee604dfc1d755b561c5b612f7d855a5d924509d890cdaccb518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T00:19:54.656 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 podman[65436]: 2024-09-11 00:19:54.654965103 +0000 UTC m=+0.377153056 container init c8584de6ddb13ee604dfc1d755b561c5b612f7d855a5d924509d890cdaccb518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:19:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.723+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feef81a8040 con 0x7feef8108fa0 2024-09-11T00:19:54.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.723+0000 7feef6000640 1 -- 172.21.15.103:0/1211204807 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7feef00ad1d0 con 0x7feef8108fa0 2024-09-11T00:19:54.726 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:54.726 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":6,"btime":"2024-09-11T00:19:52:525903+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T00:19:54.726 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 6 2024-09-11T00:19:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feedc0b5410 msgr2=0x7feedc0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feedc0b5410 0x7feedc0b78d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feee8031440 tx=0x7feee8005b50 comp rx=0 tx=0).stop 2024-09-11T00:19:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 msgr2=0x7feef81a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 0x7feef81a0b30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feef000eb10 tx=0x7feef000c2d0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 shutdown_connections 2024-09-11T00:19:54.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feedc0b5410 0x7feedc0b78d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feef8108fa0 0x7feef81a0b30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feef8108620 0x7feef819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 --2- 172.21.15.103:0/1211204807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feef8102c10 0x7feef819b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:54.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 >> 172.21.15.103:0/1211204807 conn(0x7feef80fe6c0 msgr2=0x7feef810ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:54.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 shutdown_connections 2024-09-11T00:19:54.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:54.726+0000 7feefe200640 1 -- 172.21.15.103:0/1211204807 wait complete. 2024-09-11T00:19:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:54 smithi117 ceph-mon[35525]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:55.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 podman[65436]: 2024-09-11 00:19:54.657970751 +0000 UTC m=+0.380158713 container start c8584de6ddb13ee604dfc1d755b561c5b612f7d855a5d924509d890cdaccb518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-09-11T00:19:55.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 ceph-mds[65455]: -- 172.21.15.183:0/824393721 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55962ca19860 con 0x55962c9f7000 2024-09-11T00:19:55.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: starting mds.f at 2024-09-11T00:19:55.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 bash[65436]: c8584de6ddb13ee604dfc1d755b561c5b612f7d855a5d924509d890cdaccb518 2024-09-11T00:19:55.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:19:54 smithi183 systemd[1]: Started Ceph mds.f for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1211204807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:55.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:55 smithi183 ceph-mon[35934]: Deploying daemon mds.c on smithi183 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1211204807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:55.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:55 smithi103 ceph-mon[30578]: Deploying daemon mds.c on smithi183 2024-09-11T00:19:55.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1211204807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:55.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:55.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:55.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:55.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:55.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:55 smithi117 ceph-mon[35525]: Deploying daemon mds.c on smithi183 2024-09-11T00:19:56.661 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:56 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6834/3946403859,v1:172.21.15.183:6835/3946403859] up:boot 2024-09-11T00:19:56.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:56 smithi183 ceph-mon[35934]: fsmap 6 up:standby 2024-09-11T00:19:56.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:56 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:19:56.662 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:56 smithi183 ceph-mon[35934]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:56.712 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:56 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6834/3946403859,v1:172.21.15.183:6835/3946403859] up:boot 2024-09-11T00:19:56.712 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:56 smithi103 ceph-mon[30578]: fsmap 6 up:standby 2024-09-11T00:19:56.713 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:56 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:19:56.713 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:56 smithi103 ceph-mon[30578]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:56 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6834/3946403859,v1:172.21.15.183:6835/3946403859] up:boot 2024-09-11T00:19:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:56 smithi117 ceph-mon[35525]: fsmap 6 up:standby 2024-09-11T00:19:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:56 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:19:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:56 smithi117 ceph-mon[35525]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:57.030 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:56 smithi183 systemd[1]: Starting Ceph mds.c for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:57.305 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 podman[65732]: 2024-09-11 00:19:56.950380961 +0000 UTC m=+0.015672120 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:19:57.305 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 podman[65732]: 2024-09-11 00:19:57.051241244 +0000 UTC m=+0.116532400 container create 5c800929afb3dadb1283d745d1613d867ea8c191c4decca3ec64a1804800934e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T00:19:57.305 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 podman[65732]: 2024-09-11 00:19:57.303719759 +0000 UTC m=+0.369010915 container init 5c800929afb3dadb1283d745d1613d867ea8c191c4decca3ec64a1804800934e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-11T00:19:57.690 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:19:57.780 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 podman[65732]: 2024-09-11 00:19:57.306520854 +0000 UTC m=+0.371812021 container start 5c800929afb3dadb1283d745d1613d867ea8c191c4decca3ec64a1804800934e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T00:19:57.781 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 ceph-mds[65752]: -- 172.21.15.183:0/1783443721 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f1fae9860 con 0x559f1fac7400 2024-09-11T00:19:57.781 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c[65748]: starting mds.c at 2024-09-11T00:19:57.781 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 bash[65732]: 5c800929afb3dadb1283d745d1613d867ea8c191c4decca3ec64a1804800934e 2024-09-11T00:19:57.781 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:19:57 smithi183 systemd[1]: Started Ceph mds.c for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:19:58.075 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:19:58.490 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] up:boot 2024-09-11T00:19:58.490 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: fsmap 7 up:standby 2024-09-11T00:19:58.490 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:19:58.490 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.490 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.490 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.491 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:58.491 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:58.491 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:58.491 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: Deploying daemon mds.h on smithi117 2024-09-11T00:19:58.491 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:58 smithi103 ceph-mon[30578]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:58.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 -- 172.21.15.103:0/371661477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd810b350 msgr2=0x7facd8111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 --2- 172.21.15.103:0/371661477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd810b350 0x7facd8111b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7facd0009900 tx=0x7facd002a410 comp rx=0 tx=0).stop 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 -- 172.21.15.103:0/371661477 shutdown_connections 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 --2- 172.21.15.103:0/371661477 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd810b350 0x7facd8111b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 --2- 172.21.15.103:0/371661477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd810a9d0 0x7facd810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 --2- 172.21.15.103:0/371661477 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd8104fc0 0x7facd81053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 -- 172.21.15.103:0/371661477 >> 172.21.15.103:0/371661477 conn(0x7facd81009f0 msgr2=0x7facd8102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 -- 172.21.15.103:0/371661477 shutdown_connections 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.741+0000 7facdde00640 1 -- 172.21.15.103:0/371661477 wait complete. 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 Processor -- start 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 -- start start 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8104fc0 0x7facd819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd810a9d0 0x7facd819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd810b350 0x7facd81a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7facd8114f80 con 0x7facd8104fc0 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7facd8114e00 con 0x7facd810a9d0 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.742+0000 7facdde00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7facd8115100 con 0x7facd810b350 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd810a9d0 0x7facd819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8104fc0 0x7facd819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd810a9d0 0x7facd819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48578/0 (socket says 172.21.15.103:48578) 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6000640 1 -- 172.21.15.103:0/778050315 learned_addr learned my addr 172.21.15.103:0/778050315 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd7400640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd810b350 0x7facd81a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6a00640 1 -- 172.21.15.103:0/778050315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd810b350 msgr2=0x7facd81a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6a00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd810b350 0x7facd81a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6a00640 1 -- 172.21.15.103:0/778050315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd810a9d0 msgr2=0x7facd819b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6a00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd810a9d0 0x7facd819b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6a00640 1 -- 172.21.15.103:0/778050315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd81a14e0 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.743+0000 7facd6a00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8104fc0 0x7facd819b430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7facc800eb00 tx=0x7facc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.744+0000 7facdca00640 1 -- 172.21.15.103:0/778050315 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8010430 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.744+0000 7facdca00640 1 -- 172.21.15.103:0/778050315 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc8010a80 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.744+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd81a17d0 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.744+0000 7facdca00640 1 -- 172.21.15.103:0/778050315 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8017a90 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.744+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd81a7e90 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.745+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd81053a0 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.747+0000 7facdca00640 1 -- 172.21.15.103:0/778050315 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7facc8017c50 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.748+0000 7facdca00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7facbc0b52e0 0x7facbc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.749+0000 7facdca00640 1 -- 172.21.15.103:0/778050315 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7facc80e6300 con 0x7facd8104fc0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.749+0000 7facd6000640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7facbc0b52e0 0x7facbc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:19:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.749+0000 7facd6000640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7facbc0b52e0 0x7facbc0b77a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7facd8104270 tx=0x7facc000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:19:58.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.751+0000 7facdca00640 1 -- 172.21.15.103:0/778050315 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7facc80ad650 con 0x7facd8104fc0 2024-09-11T00:19:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] up:boot 2024-09-11T00:19:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: fsmap 7 up:standby 2024-09-11T00:19:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: Deploying daemon mds.h on smithi117 2024-09-11T00:19:58.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:58 smithi183 ceph-mon[35934]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] up:boot 2024-09-11T00:19:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: fsmap 7 up:standby 2024-09-11T00:19:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:19:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:19:58.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:19:58.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:19:58.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:19:58.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: Deploying daemon mds.h on smithi117 2024-09-11T00:19:58.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:58 smithi117 ceph-mon[35525]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:19:58.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.969+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facd802e5c0 con 0x7facd8104fc0 2024-09-11T00:19:58.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.970+0000 7facdca00640 1 -- 172.21.15.103:0/778050315 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7facc80acf50 con 0x7facd8104fc0 2024-09-11T00:19:58.972 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:19:58.972 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":8,"btime":"2024-09-11T00:19:57:455868+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T00:19:58.972 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 8 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7facbc0b52e0 msgr2=0x7facbc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7facbc0b52e0 0x7facbc0b77a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7facd8104270 tx=0x7facc000b4a0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8104fc0 msgr2=0x7facd819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8104fc0 0x7facd819b430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7facc800eb00 tx=0x7facc800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 shutdown_connections 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7facbc0b52e0 0x7facbc0b77a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7facd810b350 0x7facd81a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facd810a9d0 0x7facd819b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 --2- 172.21.15.103:0/778050315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facd8104fc0 0x7facd819b430 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:19:58.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 >> 172.21.15.103:0/778050315 conn(0x7facd81009f0 msgr2=0x7facd8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:19:58.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 shutdown_connections 2024-09-11T00:19:58.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:19:58.972+0000 7facdde00640 1 -- 172.21.15.103:0/778050315 wait complete. 2024-09-11T00:19:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:19:59 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/778050315' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:59.688 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:19:59 smithi117 systemd[1]: Starting Ceph mds.h for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:19:59.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:19:59 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/778050315' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:19:59.851 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:19:59 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/778050315' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:00.023 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:19:59 smithi117 podman[64965]: 2024-09-11 00:19:59.649123759 +0000 UTC m=+0.015530730 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:20:00.023 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:19:59 smithi117 podman[64965]: 2024-09-11 00:19:59.769182225 +0000 UTC m=+0.135589194 container create 01b72f8cd469f41f376c1e3eac9216bcdcd808ac7b741323f9789e921f8ccfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:20:00.023 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:20:00 smithi117 podman[64965]: 2024-09-11 00:20:00.02216856 +0000 UTC m=+0.388575530 container init 01b72f8cd469f41f376c1e3eac9216bcdcd808ac7b741323f9789e921f8ccfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T00:20:00.437 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:20:00 smithi117 podman[64965]: 2024-09-11 00:20:00.025106987 +0000 UTC m=+0.391513968 container start 01b72f8cd469f41f376c1e3eac9216bcdcd808ac7b741323f9789e921f8ccfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T00:20:00.437 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mds[64984]: -- 172.21.15.117:0/1384554719 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561761377860 con 0x561761355400 2024-09-11T00:20:00.437 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h[64980]: starting mds.h at 2024-09-11T00:20:00.437 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:20:00 smithi117 bash[64965]: 01b72f8cd469f41f376c1e3eac9216bcdcd808ac7b741323f9789e921f8ccfcf 2024-09-11T00:20:00.437 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:20:00 smithi117 systemd[1]: Started Ceph mds.h for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:20:00.766 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: overall HEALTH_OK 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: Deploying daemon mds.i on smithi183 2024-09-11T00:20:00.767 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:00 smithi183 ceph-mon[35934]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: overall HEALTH_OK 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: Deploying daemon mds.i on smithi183 2024-09-11T00:20:00.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:00 smithi103 ceph-mon[30578]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: overall HEALTH_OK 2024-09-11T00:20:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:00.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: Deploying daemon mds.i on smithi183 2024-09-11T00:20:00.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:00 smithi117 ceph-mon[35525]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:01.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:01 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6836/2700270511,v1:172.21.15.117:6837/2700270511] up:boot 2024-09-11T00:20:01.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:01 smithi183 ceph-mon[35934]: fsmap 8 up:standby 2024-09-11T00:20:01.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:01 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:20:01.890 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:20:01.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:01 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6836/2700270511,v1:172.21.15.117:6837/2700270511] up:boot 2024-09-11T00:20:01.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:01 smithi103 ceph-mon[30578]: fsmap 8 up:standby 2024-09-11T00:20:01.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:01 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:20:01.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:01 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6836/2700270511,v1:172.21.15.117:6837/2700270511] up:boot 2024-09-11T00:20:01.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:01 smithi117 ceph-mon[35525]: fsmap 8 up:standby 2024-09-11T00:20:01.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:01 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:20:02.276 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:02.281 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 systemd[1]: Starting Ceph mds.i for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:20:02.642 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:02 smithi183 ceph-mon[35934]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:02.643 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 podman[66038]: 2024-09-11 00:20:02.254765593 +0000 UTC m=+0.015777148 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:20:02.643 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 podman[66038]: 2024-09-11 00:20:02.380939866 +0000 UTC m=+0.141951414 container create 7c771aa545ac01e25ea61ce01be46f69b2d1148e2e7e63d2e442981270d5fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2) 2024-09-11T00:20:02.643 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 podman[66038]: 2024-09-11 00:20:02.642089258 +0000 UTC m=+0.403100813 container init 7c771aa545ac01e25ea61ce01be46f69b2d1148e2e7e63d2e442981270d5fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T00:20:02.667 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:02 smithi103 ceph-mon[30578]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:02 smithi117 ceph-mon[35525]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:02.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 -- 172.21.15.103:0/4119229847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 msgr2=0x7f26f810f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:02.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 --2- 172.21.15.103:0/4119229847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 0x7f26f810f790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009900 tx=0x7f26f002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:20:02.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 -- 172.21.15.103:0/4119229847 shutdown_connections 2024-09-11T00:20:02.956 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 --2- 172.21.15.103:0/4119229847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 0x7f26f810f790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 --2- 172.21.15.103:0/4119229847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f8108610 0x7f26f8108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 --2- 172.21.15.103:0/4119229847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 0x7f26f8102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 -- 172.21.15.103:0/4119229847 >> 172.21.15.103:0/4119229847 conn(0x7f26f80fe6d0 msgr2=0x7f26f8100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 -- 172.21.15.103:0/4119229847 shutdown_connections 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.934+0000 7f26fe000640 1 -- 172.21.15.103:0/4119229847 wait complete. 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 Processor -- start 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 -- start start 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 0x7f26f819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f8108610 0x7f26f819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:02.957 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 0x7f26f81a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8112c70 con 0x7f26f8102c00 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8112af0 con 0x7f26f8108610 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fe000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26f8112df0 con 0x7f26f8108f90 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.935+0000 7f26fcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 0x7f26f819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 0x7f26f819b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59008/0 (socket says 172.21.15.103:59008) 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 -- 172.21.15.103:0/1536333725 learned_addr learned my addr 172.21.15.103:0/1536333725 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26f7e00640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f8108610 0x7f26f819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fd600640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 0x7f26f81a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 -- 172.21.15.103:0/1536333725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 msgr2=0x7f26f81a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 0x7f26f81a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 -- 172.21.15.103:0/1536333725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f8108610 msgr2=0x7f26f819b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:02.958 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f8108610 0x7f26f819b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 -- 172.21.15.103:0/1536333725 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f81a1270 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26f7e00640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f8108610 0x7f26f819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.936+0000 7f26fcc00640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 0x7f26f819b1c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f26e800ebc0 tx=0x7f26e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.937+0000 7f26f5600640 1 -- 172.21.15.103:0/1536333725 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8010630 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.937+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f81a1560 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.937+0000 7f26f5600640 1 -- 172.21.15.103:0/1536333725 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e8010c80 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.937+0000 7f26f5600640 1 -- 172.21.15.103:0/1536333725 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8018b60 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.937+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f81a7c20 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.938+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8102fe0 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.940+0000 7f26f5600640 1 -- 172.21.15.103:0/1536333725 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f26e80107d0 con 0x7f26f8102c00 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.942+0000 7f26f5600640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f26dc0b5390 0x7f26dc0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:02.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.942+0000 7f26f5600640 1 -- 172.21.15.103:0/1536333725 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f26e80e5da0 con 0x7f26f8102c00 2024-09-11T00:20:02.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.942+0000 7f26f7e00640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f26dc0b5390 0x7f26dc0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:02.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.943+0000 7f26f7e00640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f26dc0b5390 0x7f26dc0b7850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26f8104ec0 tx=0x7f26e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:02.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:02.946+0000 7f26f5600640 1 -- 172.21.15.103:0/1536333725 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f26e80aca10 con 0x7f26f8102c00 2024-09-11T00:20:03.030 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 podman[66038]: 2024-09-11 00:20:02.645097929 +0000 UTC m=+0.406109481 container start 7c771aa545ac01e25ea61ce01be46f69b2d1148e2e7e63d2e442981270d5fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T00:20:03.030 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 ceph-mds[66057]: -- 172.21.15.183:0/1099765805 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593739e1860 con 0x5593739be800 2024-09-11T00:20:03.031 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i[66053]: starting mds.i at 2024-09-11T00:20:03.031 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 bash[66038]: 7c771aa545ac01e25ea61ce01be46f69b2d1148e2e7e63d2e442981270d5fcd3 2024-09-11T00:20:03.031 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:20:02 smithi183 systemd[1]: Started Ceph mds.i for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:20:03.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.178+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26f802e5c0 con 0x7f26f8102c00 2024-09-11T00:20:03.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.179+0000 7f26f5600640 1 -- 172.21.15.103:0/1536333725 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7228 (secure 0 0 0) 0x7f26e80b1440 con 0x7f26f8102c00 2024-09-11T00:20:03.181 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:03.181 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":9,"btime":"2024-09-11T00:20:00:499184+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-11T00:20:03.182 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 9 2024-09-11T00:20:03.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f26dc0b5390 msgr2=0x7f26dc0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f26dc0b5390 0x7f26dc0b7850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26f8104ec0 tx=0x7f26e0005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 msgr2=0x7f26f819b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 0x7f26f819b1c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f26e800ebc0 tx=0x7f26e800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 shutdown_connections 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f26dc0b5390 0x7f26dc0b7850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f26f8108f90 0x7f26f81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f8108610 0x7f26f819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 --2- 172.21.15.103:0/1536333725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26f8102c00 0x7f26f819b1c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.181+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 >> 172.21.15.103:0/1536333725 conn(0x7f26f80fe6d0 msgr2=0x7f26f8075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:03.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.182+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 shutdown_connections 2024-09-11T00:20:03.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:03.182+0000 7f26fe000640 1 -- 172.21.15.103:0/1536333725 wait complete. 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: Deploying daemon mds.g on smithi103 2024-09-11T00:20:04.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1536333725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:04.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] up:boot 2024-09-11T00:20:04.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: fsmap 9 up:standby 2024-09-11T00:20:04.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:03 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:20:04.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:04.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:04.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:04.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: Deploying daemon mds.g on smithi103 2024-09-11T00:20:04.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1536333725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:04.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] up:boot 2024-09-11T00:20:04.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: fsmap 9 up:standby 2024-09-11T00:20:04.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:03 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: Deploying daemon mds.g on smithi103 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1536333725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:04.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] up:boot 2024-09-11T00:20:04.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: fsmap 9 up:standby 2024-09-11T00:20:04.282 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:03 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:20:05.163 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:04 smithi103 ceph-mon[30578]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:05.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:04 smithi117 ceph-mon[35525]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:05.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:04 smithi183 ceph-mon[35934]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:06.237 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:20:06.579 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:06 smithi103 ceph-mon[30578]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:06.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:06 smithi183 ceph-mon[35934]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:06.920 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:20:06 smithi103 podman[72842]: 2024-09-11 00:20:06.615330574 +0000 UTC m=+0.161625898 container create c198b4477c4ed00c74c56059c78fe300f1756bd873e4f3c21e78649753e4fa3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T00:20:06.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:06 smithi117 ceph-mon[35525]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:07.175 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:07.317 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:20:06 smithi103 podman[72842]: 2024-09-11 00:20:06.943732383 +0000 UTC m=+0.490027708 container init c198b4477c4ed00c74c56059c78fe300f1756bd873e4f3c21e78649753e4fa3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T00:20:07.317 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:20:06 smithi103 podman[72842]: 2024-09-11 00:20:06.950195838 +0000 UTC m=+0.496491163 container start c198b4477c4ed00c74c56059c78fe300f1756bd873e4f3c21e78649753e4fa3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD) 2024-09-11T00:20:07.317 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:20:06 smithi103 ceph-mds[72873]: -- 172.21.15.103:0/3245568877 <== mon.0 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588695e9860 con 0x5588695c7400 2024-09-11T00:20:07.317 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:20:06 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g[72869]: starting mds.g at 2024-09-11T00:20:07.317 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:20:07 smithi103 bash[72842]: c198b4477c4ed00c74c56059c78fe300f1756bd873e4f3c21e78649753e4fa3e 2024-09-11T00:20:07.317 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:20:07 smithi103 systemd[1]: Started Ceph mds.g for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:20:07.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 -- 172.21.15.103:0/1212111039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4410a800 msgr2=0x7f2e4410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:07.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1212111039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4410a800 0x7f2e4410ac40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009900 tx=0x7f2e3002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:20:07.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 -- 172.21.15.103:0/1212111039 shutdown_connections 2024-09-11T00:20:07.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1212111039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4410b180 0x7f2e44111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:07.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1212111039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e4410a800 0x7f2e4410ac40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:07.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1212111039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e44104df0 0x7f2e441051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:07.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 -- 172.21.15.103:0/1212111039 >> 172.21.15.103:0/1212111039 conn(0x7f2e44100840 msgr2=0x7f2e44102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 -- 172.21.15.103:0/1212111039 shutdown_connections 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.846+0000 7f2e4b000640 1 -- 172.21.15.103:0/1212111039 wait complete. 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.847+0000 7f2e4b000640 1 Processor -- start 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.847+0000 7f2e4b000640 1 -- start start 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.847+0000 7f2e4b000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e44104df0 0x7f2e4419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.847+0000 7f2e4b000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4410a800 0x7f2e4419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e4b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4410b180 0x7f2e441a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e4b000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44114e90 con 0x7f2e44104df0 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e4b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44114d10 con 0x7f2e4410b180 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e4b000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44115010 con 0x7f2e4410a800 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e44104df0 0x7f2e4419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:07.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4410a800 0x7f2e4419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4410a800 0x7f2e4419b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:48120/0 (socket says 172.21.15.103:48120) 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 -- 172.21.15.103:0/1917564436 learned_addr learned my addr 172.21.15.103:0/1917564436 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e4a600640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4410b180 0x7f2e441a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 -- 172.21.15.103:0/1917564436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4410b180 msgr2=0x7f2e441a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4410b180 0x7f2e441a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 -- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e44104df0 msgr2=0x7f2e4419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e44104df0 0x7f2e4419b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e49200640 1 -- 172.21.15.103:0/1917564436 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441a1310 con 0x7f2e4410a800 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.848+0000 7f2e4a600640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4410b180 0x7f2e441a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.849+0000 7f2e49200640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4410a800 0x7f2e4419b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e30005ab0 tx=0x7f2e30003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.849+0000 7f2e42000640 1 -- 172.21.15.103:0/1917564436 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3003f070 con 0x7f2e4410a800 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.849+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441a15a0 con 0x7f2e4410a800 2024-09-11T00:20:07.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.849+0000 7f2e42000640 1 -- 172.21.15.103:0/1917564436 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e30031b60 con 0x7f2e4410a800 2024-09-11T00:20:07.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.849+0000 7f2e42000640 1 -- 172.21.15.103:0/1917564436 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30035030 con 0x7f2e4410a800 2024-09-11T00:20:07.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.849+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a7cc0 con 0x7f2e4410a800 2024-09-11T00:20:07.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.850+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e441051d0 con 0x7f2e4410a800 2024-09-11T00:20:07.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.852+0000 7f2e42000640 1 -- 172.21.15.103:0/1917564436 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f2e3003b460 con 0x7f2e4410a800 2024-09-11T00:20:07.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.854+0000 7f2e42000640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2e2c0b5390 0x7f2e2c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:07.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.855+0000 7f2e42000640 1 -- 172.21.15.103:0/1917564436 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2e301063d0 con 0x7f2e4410a800 2024-09-11T00:20:07.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.855+0000 7f2e49c00640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2e2c0b5390 0x7f2e2c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:07.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.855+0000 7f2e49c00640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2e2c0b5390 0x7f2e2c0b7850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009bd0 tx=0x7f2e38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:07.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:07.856+0000 7f2e42000640 1 -- 172.21.15.103:0/1917564436 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2e300cd040 con 0x7f2e4410a800 2024-09-11T00:20:08.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.098+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e441a7f50 con 0x7f2e4410a800 2024-09-11T00:20:08.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.099+0000 7f2e42000640 1 -- 172.21.15.103:0/1917564436 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8884 (secure 0 0 0) 0x7f2e300d1a70 con 0x7f2e4410a800 2024-09-11T00:20:08.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:08.101 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":11,"btime":"2024-09-11T00:20:07:447677+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24643,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T00:20:08.101 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 11 2024-09-11T00:20:08.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2e2c0b5390 msgr2=0x7f2e2c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:08.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2e2c0b5390 0x7f2e2c0b7850 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009bd0 tx=0x7f2e38005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4410a800 msgr2=0x7f2e4419b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4410a800 0x7f2e4419b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2e30005ab0 tx=0x7f2e30003c10 comp rx=0 tx=0).stop 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 shutdown_connections 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2e2c0b5390 0x7f2e2c0b7850 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e4410b180 0x7f2e441a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2e4410a800 0x7f2e4419b7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 --2- 172.21.15.103:0/1917564436 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2e44104df0 0x7f2e4419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 >> 172.21.15.103:0/1917564436 conn(0x7f2e44100840 msgr2=0x7f2e44103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 shutdown_connections 2024-09-11T00:20:08.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:08.101+0000 7f2e4b000640 1 -- 172.21.15.103:0/1917564436 wait complete. 2024-09-11T00:20:08.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:08.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:08.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:08.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: Deploying daemon mds.a on smithi103 2024-09-11T00:20:08.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.103:6838/3371048612,v1:172.21.15.103:6839/3371048612] up:boot 2024-09-11T00:20:08.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: fsmap 10 up:standby 2024-09-11T00:20:08.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:20:08.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:08 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1917564436' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:08.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:08.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:08.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:08.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: Deploying daemon mds.a on smithi103 2024-09-11T00:20:08.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.103:6838/3371048612,v1:172.21.15.103:6839/3371048612] up:boot 2024-09-11T00:20:08.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: fsmap 10 up:standby 2024-09-11T00:20:08.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:20:08.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:08 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1917564436' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:08.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: Deploying daemon mds.a on smithi103 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.103:6838/3371048612,v1:172.21.15.103:6839/3371048612] up:boot 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: fsmap 10 up:standby 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-11T00:20:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:08 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1917564436' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:09.251 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:09 smithi103 ceph-mon[30578]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:09.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:09 smithi117 ceph-mon[35525]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:09.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:09 smithi183 ceph-mon[35934]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:10.703 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:10 smithi103 ceph-mon[30578]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:10 smithi183 ceph-mon[35934]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:10 smithi117 ceph-mon[35525]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:10.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:20:11.017 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:10 smithi103 systemd[1]: Starting Ceph mds.a for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:20:11.335 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 podman[73235]: 2024-09-11 00:20:10.920897241 +0000 UTC m=+0.027103119 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:20:11.335 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 podman[73235]: 2024-09-11 00:20:11.066481589 +0000 UTC m=+0.172687473 container create 6151c5100799f3162206b79dba8d2d72fca96e8bab599940319fb561e9c456e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T00:20:11.335 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 podman[73235]: 2024-09-11 00:20:11.334266545 +0000 UTC m=+0.440472424 container init 6151c5100799f3162206b79dba8d2d72fca96e8bab599940319fb561e9c456e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T00:20:11.548 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:11.670 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 podman[73235]: 2024-09-11 00:20:11.342202028 +0000 UTC m=+0.448407916 container start 6151c5100799f3162206b79dba8d2d72fca96e8bab599940319fb561e9c456e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T00:20:11.670 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 ceph-mds[73283]: -- 172.21.15.103:0/2518719977 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577a5371860 con 0x5577a534f400 2024-09-11T00:20:11.670 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a[73267]: starting mds.a at 2024-09-11T00:20:11.670 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 bash[73235]: 6151c5100799f3162206b79dba8d2d72fca96e8bab599940319fb561e9c456e9 2024-09-11T00:20:11.670 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:20:11 smithi103 systemd[1]: Started Ceph mds.a for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:20:12.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 -- 172.21.15.103:0/2749113624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c8102da0 msgr2=0x7fe1c8103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2749113624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c8102da0 0x7fe1c8103180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009900 tx=0x7fe1b802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 -- 172.21.15.103:0/2749113624 shutdown_connections 2024-09-11T00:20:12.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2749113624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c8109130 0x7fe1c810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2749113624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c81087b0 0x7fe1c8108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2749113624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c8102da0 0x7fe1c8103180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 -- 172.21.15.103:0/2749113624 >> 172.21.15.103:0/2749113624 conn(0x7fe1c80fe810 msgr2=0x7fe1c8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 -- 172.21.15.103:0/2749113624 shutdown_connections 2024-09-11T00:20:12.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.166+0000 7fe1cea00640 1 -- 172.21.15.103:0/2749113624 wait complete. 2024-09-11T00:20:12.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.167+0000 7fe1cea00640 1 Processor -- start 2024-09-11T00:20:12.170 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.167+0000 7fe1cea00640 1 -- start start 2024-09-11T00:20:12.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1cea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c8102da0 0x7fe1c819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1cea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c81087b0 0x7fe1c819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1cea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c8109130 0x7fe1c81a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1cea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c8112e40 con 0x7fe1c8102da0 2024-09-11T00:20:12.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1cea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c8112cc0 con 0x7fe1c81087b0 2024-09-11T00:20:12.171 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1cea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c8112fc0 con 0x7fe1c8109130 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c8102da0 0x7fe1c819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c8102da0 0x7fe1c819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34872/0 (socket says 172.21.15.103:34872) 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1cca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c8109130 0x7fe1c81a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7e00640 1 -- 172.21.15.103:0/2318516765 learned_addr learned my addr 172.21.15.103:0/2318516765 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7400640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c81087b0 0x7fe1c819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7400640 1 -- 172.21.15.103:0/2318516765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c8109130 msgr2=0x7fe1c81a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7400640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c8109130 0x7fe1c81a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7400640 1 -- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c8102da0 msgr2=0x7fe1c819b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7400640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c8102da0 0x7fe1c819b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.168+0000 7fe1c7400640 1 -- 172.21.15.103:0/2318516765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c81a1550 con 0x7fe1c81087b0 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.169+0000 7fe1c7e00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c8102da0 0x7fe1c819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.169+0000 7fe1c7400640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c81087b0 0x7fe1c819b9e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9f0 tx=0x7fe1b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.172 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.169+0000 7fe1c4c00640 1 -- 172.21.15.103:0/2318516765 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b400cd80 con 0x7fe1c81087b0 2024-09-11T00:20:12.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.169+0000 7fe1c4c00640 1 -- 172.21.15.103:0/2318516765 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b40035f0 con 0x7fe1c81087b0 2024-09-11T00:20:12.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.169+0000 7fe1c4c00640 1 -- 172.21.15.103:0/2318516765 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4010670 con 0x7fe1c81087b0 2024-09-11T00:20:12.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.169+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c81a1840 con 0x7fe1c81087b0 2024-09-11T00:20:12.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.169+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c81a7f00 con 0x7fe1c81087b0 2024-09-11T00:20:12.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.170+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c8103180 con 0x7fe1c81087b0 2024-09-11T00:20:12.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.173+0000 7fe1c4c00640 1 -- 172.21.15.103:0/2318516765 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fe1b4004000 con 0x7fe1c81087b0 2024-09-11T00:20:12.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.174+0000 7fe1c4c00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1ac0b52e0 0x7fe1ac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:12.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.175+0000 7fe1c4c00640 1 -- 172.21.15.103:0/2318516765 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe1b40e5a30 con 0x7fe1c81087b0 2024-09-11T00:20:12.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.175+0000 7fe1c7e00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1ac0b52e0 0x7fe1ac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:12.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.175+0000 7fe1c7e00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1ac0b52e0 0x7fe1ac0b77a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80096c0 tx=0x7fe1b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:12.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.178+0000 7fe1c4c00640 1 -- 172.21.15.103:0/2318516765 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe1b40accf0 con 0x7fe1c81087b0 2024-09-11T00:20:12.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.398+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1c802e5c0 con 0x7fe1c81087b0 2024-09-11T00:20:12.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.399+0000 7fe1c4c00640 1 -- 172.21.15.103:0/2318516765 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7fe1b40ac5f0 con 0x7fe1c81087b0 2024-09-11T00:20:12.401 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:12.401 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":12,"btime":"2024-09-11T00:20:11:448188+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24643,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-11T00:20:12.401 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 12 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1ac0b52e0 msgr2=0x7fe1ac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1ac0b52e0 0x7fe1ac0b77a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80096c0 tx=0x7fe1b8005a20 comp rx=0 tx=0).stop 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c81087b0 msgr2=0x7fe1c819b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c81087b0 0x7fe1c819b9e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400e9f0 tx=0x7fe1b400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 shutdown_connections 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1ac0b52e0 0x7fe1ac0b77a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1c8109130 0x7fe1c81a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1c81087b0 0x7fe1c819b9e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 --2- 172.21.15.103:0/2318516765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1c8102da0 0x7fe1c819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:12.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 >> 172.21.15.103:0/2318516765 conn(0x7fe1c80fe810 msgr2=0x7fe1c8104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:12.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 shutdown_connections 2024-09-11T00:20:12.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:12.401+0000 7fe1cea00640 1 -- 172.21.15.103:0/2318516765 wait complete. 2024-09-11T00:20:12.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.103:6840/4229734989,v1:172.21.15.103:6841/4229734989] up:boot 2024-09-11T00:20:12.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: fsmap 11 up:standby 2024-09-11T00:20:12.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:20:12.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.537 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:12.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:12.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:12.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: Deploying daemon mds.b on smithi117 2024-09-11T00:20:12.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:12.538 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:12 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2318516765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.103:6840/4229734989,v1:172.21.15.103:6841/4229734989] up:boot 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: fsmap 11 up:standby 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: Deploying daemon mds.b on smithi117 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:12 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2318516765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:12.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.103:6840/4229734989,v1:172.21.15.103:6841/4229734989] up:boot 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: fsmap 11 up:standby 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: Deploying daemon mds.b on smithi117 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:12.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:12 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2318516765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:13.687 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:13 smithi117 systemd[1]: Starting Ceph mds.b for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:20:14.051 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:13 smithi117 podman[65278]: 2024-09-11 00:20:13.688824009 +0000 UTC m=+0.016092738 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:20:14.051 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:13 smithi117 podman[65278]: 2024-09-11 00:20:13.805294086 +0000 UTC m=+0.132562805 container create 0c50953b2ef50d89f4b250a19e2bedae23bca2c1b5b514a64710a99f5d274be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T00:20:14.051 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:14 smithi117 podman[65278]: 2024-09-11 00:20:14.049840169 +0000 UTC m=+0.377108888 container init 0c50953b2ef50d89f4b250a19e2bedae23bca2c1b5b514a64710a99f5d274be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T00:20:14.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:14 smithi117 podman[65278]: 2024-09-11 00:20:14.053129067 +0000 UTC m=+0.380397787 container start 0c50953b2ef50d89f4b250a19e2bedae23bca2c1b5b514a64710a99f5d274be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:20:14.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:14 smithi117 ceph-mds[65297]: -- 172.21.15.117:0/4071672243 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55916615b860 con 0x559166139400 2024-09-11T00:20:14.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:14 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: starting mds.b at 2024-09-11T00:20:14.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:14 smithi117 bash[65278]: 0c50953b2ef50d89f4b250a19e2bedae23bca2c1b5b514a64710a99f5d274be1 2024-09-11T00:20:14.438 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:14 smithi117 systemd[1]: Started Ceph mds.b for 40f10072-6fd2-11ef-bcea-c7b262605968. 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] up:boot 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: fsmap 12 up:standby 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.280 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.281 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:15.281 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:15 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:15.375 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:20:15.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] up:boot 2024-09-11T00:20:15.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: fsmap 12 up:standby 2024-09-11T00:20:15.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:20:15.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:15.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:15 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:15.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] up:boot 2024-09-11T00:20:15.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: fsmap 12 up:standby 2024-09-11T00:20:15.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:20:15.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:15.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:15.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:15 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:15.946 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:16.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 -- 172.21.15.103:0/392427151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabe810a9b0 msgr2=0x7fabe810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:16.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 --2- 172.21.15.103:0/392427151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabe810a9b0 0x7fabe810adf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009900 tx=0x7fabd402a410 comp rx=0 tx=0).stop 2024-09-11T00:20:16.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 -- 172.21.15.103:0/392427151 shutdown_connections 2024-09-11T00:20:16.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 --2- 172.21.15.103:0/392427151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe810b330 0x7fabe8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 --2- 172.21.15.103:0/392427151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabe810a9b0 0x7fabe810adf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 --2- 172.21.15.103:0/392427151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe8104fa0 0x7fabe8105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 -- 172.21.15.103:0/392427151 >> 172.21.15.103:0/392427151 conn(0x7fabe8100a50 msgr2=0x7fabe8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:16.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 -- 172.21.15.103:0/392427151 shutdown_connections 2024-09-11T00:20:16.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.613+0000 7fabefa00640 1 -- 172.21.15.103:0/392427151 wait complete. 2024-09-11T00:20:16.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 Processor -- start 2024-09-11T00:20:16.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 -- start start 2024-09-11T00:20:16.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8104fa0 0x7fabe819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:16.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe810a9b0 0x7fabe819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:16.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabe810b330 0x7fabe81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:16.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8114f80 con 0x7fabe810b330 2024-09-11T00:20:16.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8114e00 con 0x7fabe810a9b0 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.614+0000 7fabefa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8115100 con 0x7fabe8104fa0 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8104fa0 0x7fabe819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8104fa0 0x7fabe819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:52966/0 (socket says 172.21.15.103:52966) 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 -- 172.21.15.103:0/2916762365 learned_addr learned my addr 172.21.15.103:0/2916762365 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 -- 172.21.15.103:0/2916762365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe810a9b0 msgr2=0x7fabe819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe810a9b0 0x7fabe819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 -- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabe810b330 msgr2=0x7fabe81a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabe810b330 0x7fabe81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.615+0000 7fabed000640 1 -- 172.21.15.103:0/2916762365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe81a15a0 con 0x7fabe8104fa0 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.616+0000 7fabed000640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8104fa0 0x7fabe819b4f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009f50 tx=0x7fabdc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.616+0000 7fabe5600640 1 -- 172.21.15.103:0/2916762365 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc024070 con 0x7fabe8104fa0 2024-09-11T00:20:16.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.616+0000 7fabe5600640 1 -- 172.21.15.103:0/2916762365 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabdc01f440 con 0x7fabe8104fa0 2024-09-11T00:20:16.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.616+0000 7fabe5600640 1 -- 172.21.15.103:0/2916762365 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00ce60 con 0x7fabe8104fa0 2024-09-11T00:20:16.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.616+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe81a1890 con 0x7fabe8104fa0 2024-09-11T00:20:16.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.616+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe8077760 con 0x7fabe8104fa0 2024-09-11T00:20:16.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.617+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe8106250 con 0x7fabe8104fa0 2024-09-11T00:20:16.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.620+0000 7fabe5600640 1 -- 172.21.15.103:0/2916762365 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fabdc00c9b0 con 0x7fabe8104fa0 2024-09-11T00:20:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.621+0000 7fabe5600640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fabd00b5360 0x7fabd00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.621+0000 7fabe5600640 1 -- 172.21.15.103:0/2916762365 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fabdc0ee100 con 0x7fabe8104fa0 2024-09-11T00:20:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.621+0000 7fabe7e00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fabd00b5360 0x7fabd00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:16.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.622+0000 7fabe7e00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fabd00b5360 0x7fabd00b7820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fabd402a930 tx=0x7fabd4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:16.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.622+0000 7fabe5600640 1 -- 172.21.15.103:0/2916762365 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fabdc0b53c0 con 0x7fabe8104fa0 2024-09-11T00:20:16.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.874+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabe810d140 con 0x7fabe8104fa0 2024-09-11T00:20:16.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.879+0000 7fabe5600640 1 -- 172.21.15.103:0/2916762365 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7fabdc0b4cc0 con 0x7fabe8104fa0 2024-09-11T00:20:16.881 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:16.881 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":13,"btime":"2024-09-11T00:20:14:131175+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24643,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24656,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T00:20:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 13 2024-09-11T00:20:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fabd00b5360 msgr2=0x7fabd00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fabd00b5360 0x7fabd00b7820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fabd402a930 tx=0x7fabd4031400 comp rx=0 tx=0).stop 2024-09-11T00:20:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8104fa0 msgr2=0x7fabe819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:16.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8104fa0 0x7fabe819b4f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009f50 tx=0x7fabdc00ee50 comp rx=0 tx=0).stop 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 shutdown_connections 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fabd00b5360 0x7fabd00b7820 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabe810b330 0x7fabe81a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabe810a9b0 0x7fabe819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 --2- 172.21.15.103:0/2916762365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabe8104fa0 0x7fabe819b4f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 >> 172.21.15.103:0/2916762365 conn(0x7fabe8100a50 msgr2=0x7fabe8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 shutdown_connections 2024-09-11T00:20:16.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:16.883+0000 7fabefa00640 1 -- 172.21.15.103:0/2916762365 wait complete. 2024-09-11T00:20:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:16 smithi103 ceph-mon[30578]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:17.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:16 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:16 smithi117 ceph-mon[35525]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:17.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:16 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:16 smithi183 ceph-mon[35934]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:17.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:16 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:17.874 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T00:20:18.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:17 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2916762365' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:17 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:17 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2916762365' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:17 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:18.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:17 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2916762365' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:18.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:18.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:18.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:18.252 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:17 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:20:19.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:18 smithi103 ceph-mon[30578]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:19.178 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:18 smithi117 ceph-mon[35525]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:19.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:18 smithi183 ceph-mon[35934]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:19.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.806+0000 7feea5c00640 1 -- 172.21.15.103:0/1413687263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010a800 msgr2=0x7feea010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:19.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.806+0000 7feea5c00640 1 --2- 172.21.15.103:0/1413687263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010a800 0x7feea010ac40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009860 tx=0x7fee8c02a340 comp rx=0 tx=0).stop 2024-09-11T00:20:19.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.807+0000 7feea5c00640 1 -- 172.21.15.103:0/1413687263 shutdown_connections 2024-09-11T00:20:19.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.807+0000 7feea5c00640 1 --2- 172.21.15.103:0/1413687263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feea010b180 0x7feea0111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:19.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.807+0000 7feea5c00640 1 --2- 172.21.15.103:0/1413687263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010a800 0x7feea010ac40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:19.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.807+0000 7feea5c00640 1 --2- 172.21.15.103:0/1413687263 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feea0104df0 0x7feea01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:19.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.807+0000 7feea5c00640 1 -- 172.21.15.103:0/1413687263 >> 172.21.15.103:0/1413687263 conn(0x7feea0100840 msgr2=0x7feea0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:19.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.807+0000 7feea5c00640 1 -- 172.21.15.103:0/1413687263 shutdown_connections 2024-09-11T00:20:19.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.807+0000 7feea5c00640 1 -- 172.21.15.103:0/1413687263 wait complete. 2024-09-11T00:20:19.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 Processor -- start 2024-09-11T00:20:19.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 -- start start 2024-09-11T00:20:19.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feea0104df0 0x7feea019b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:19.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feea010a800 0x7feea019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:19.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010b180 0x7feea01a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:19.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feea0114e40 con 0x7feea010a800 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feea0114cc0 con 0x7feea010b180 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7feea5c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feea0114fc0 con 0x7feea0104df0 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7fee9fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feea0104df0 0x7feea019b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.808+0000 7fee9f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feea010a800 0x7feea019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7fee9f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feea010a800 0x7feea019b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34916/0 (socket says 172.21.15.103:34916) 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7fee9fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feea0104df0 0x7feea019b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:52982/0 (socket says 172.21.15.103:52982) 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010b180 0x7feea01a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7fee9f400640 1 -- 172.21.15.103:0/2761828158 learned_addr learned my addr 172.21.15.103:0/2761828158 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:19.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5200640 1 -- 172.21.15.103:0/2761828158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feea0104df0 msgr2=0x7feea019b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5200640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feea0104df0 0x7feea019b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5200640 1 -- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feea010a800 msgr2=0x7feea019b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5200640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feea010a800 0x7feea019b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5200640 1 -- 172.21.15.103:0/2761828158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea01a1280 con 0x7feea010b180 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5200640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010b180 0x7feea01a0ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee98005fd0 tx=0x7fee9800ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7fee9cc00640 1 -- 172.21.15.103:0/2761828158 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9800ccc0 con 0x7feea010b180 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7fee9cc00640 1 -- 172.21.15.103:0/2761828158 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee980035f0 con 0x7feea010b180 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7fee9cc00640 1 -- 172.21.15.103:0/2761828158 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9800ce60 con 0x7feea010b180 2024-09-11T00:20:19.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.809+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea01a1570 con 0x7feea010b180 2024-09-11T00:20:19.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.810+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea01a7c30 con 0x7feea010b180 2024-09-11T00:20:19.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.811+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea01051d0 con 0x7feea010b180 2024-09-11T00:20:19.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.813+0000 7fee9cc00640 1 -- 172.21.15.103:0/2761828158 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fee98004000 con 0x7feea010b180 2024-09-11T00:20:19.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.814+0000 7fee9cc00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fee840b5390 0x7fee840b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:19.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.814+0000 7fee9cc00640 1 -- 172.21.15.103:0/2761828158 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fee980e5ac0 con 0x7feea010b180 2024-09-11T00:20:19.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.814+0000 7fee9fe00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fee840b5390 0x7fee840b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:19.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.815+0000 7fee9fe00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fee840b5390 0x7fee840b7850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee90005fd0 tx=0x7fee90009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:19.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:19.818+0000 7fee9cc00640 1 -- 172.21.15.103:0/2761828158 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fee98009510 con 0x7feea010b180 2024-09-11T00:20:20.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.019+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feea01a7fd0 con 0x7feea010b180 2024-09-11T00:20:20.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.021 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:19 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.020+0000 7fee9cc00640 1 -- 172.21.15.103:0/2761828158 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7fee980acd80 con 0x7feea010b180 2024-09-11T00:20:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:20.022 INFO:teuthology.orchestra.run.smithi103.stdout:[ 2024-09-11T00:20:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:20:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: "poolnum": 1, 2024-09-11T00:20:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: "poolname": ".mgr" 2024-09-11T00:20:20.022 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:20:20.022 INFO:teuthology.orchestra.run.smithi103.stdout:] 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.023+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fee840b5390 msgr2=0x7fee840b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.023+0000 7feea5c00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fee840b5390 0x7fee840b7850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fee90005fd0 tx=0x7fee90009500 comp rx=0 tx=0).stop 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010b180 msgr2=0x7feea01a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010b180 0x7feea01a0ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee98005fd0 tx=0x7fee9800ed20 comp rx=0 tx=0).stop 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 shutdown_connections 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fee840b5390 0x7fee840b7850 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feea010b180 0x7feea01a0ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feea010a800 0x7feea019b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 --2- 172.21.15.103:0/2761828158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feea0104df0 0x7feea019b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 >> 172.21.15.103:0/2761828158 conn(0x7feea0100840 msgr2=0x7feea0103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:20.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 shutdown_connections 2024-09-11T00:20:20.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:20.024+0000 7feea5c00640 1 -- 172.21.15.103:0/2761828158 wait complete. 2024-09-11T00:20:20.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:19 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:19 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:20.949 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T00:20:20.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T00:20:21.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:20 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2761828158' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:20:21.172 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:20 smithi103 ceph-mon[30578]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:21.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:20 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2761828158' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:20:21.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:20 smithi183 ceph-mon[35934]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:21.334 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:20 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2761828158' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-11T00:20:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:20 smithi117 ceph-mon[35525]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 -- 172.21.15.103:0/2786366275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 msgr2=0x7f0ca010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 --2- 172.21.15.103:0/2786366275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 0x7f0ca010ac60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009900 tx=0x7f0c8802a410 comp rx=0 tx=0).stop 2024-09-11T00:20:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 -- 172.21.15.103:0/2786366275 shutdown_connections 2024-09-11T00:20:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 --2- 172.21.15.103:0/2786366275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca010b1a0 0x7f0ca01119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 --2- 172.21.15.103:0/2786366275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 0x7f0ca010ac60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 --2- 172.21.15.103:0/2786366275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca0104e10 0x7f0ca01051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:21.907 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 -- 172.21.15.103:0/2786366275 >> 172.21.15.103:0/2786366275 conn(0x7f0ca0100840 msgr2=0x7f0ca0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 -- 172.21.15.103:0/2786366275 shutdown_connections 2024-09-11T00:20:21.908 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.905+0000 7f0ca6200640 1 -- 172.21.15.103:0/2786366275 wait complete. 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 Processor -- start 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 -- start start 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca0104e10 0x7f0ca019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 0x7f0ca019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca010b1a0 0x7f0ca01a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca0114e70 con 0x7f0ca010a820 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca0114cf0 con 0x7f0ca010b1a0 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca6200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca0114ff0 con 0x7f0ca0104e10 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0c97e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 0x7f0ca019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca4e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca0104e10 0x7f0ca019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:21.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca4e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca0104e10 0x7f0ca019b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:53006/0 (socket says 172.21.15.103:53006) 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0c97e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 0x7f0ca019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34932/0 (socket says 172.21.15.103:34932) 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca4e00640 1 -- 172.21.15.103:0/3571460756 learned_addr learned my addr 172.21.15.103:0/3571460756 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.907+0000 7f0ca5800640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca010b1a0 0x7f0ca01a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0ca4e00640 1 -- 172.21.15.103:0/3571460756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca010b1a0 msgr2=0x7f0ca01a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0ca4e00640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca010b1a0 0x7f0ca01a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0ca4e00640 1 -- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 msgr2=0x7f0ca019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0ca4e00640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 0x7f0ca019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0ca4e00640 1 -- 172.21.15.103:0/3571460756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca01a12d0 con 0x7f0ca0104e10 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0ca4e00640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca0104e10 0x7f0ca019b220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e9f0 tx=0x7f0c9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0c9e000640 1 -- 172.21.15.103:0/3571460756 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9000cd80 con 0x7f0ca0104e10 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca01a15c0 con 0x7f0ca0104e10 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0c9e000640 1 -- 172.21.15.103:0/3571460756 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c900108e0 con 0x7f0ca0104e10 2024-09-11T00:20:21.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.908+0000 7f0c9e000640 1 -- 172.21.15.103:0/3571460756 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90018980 con 0x7f0ca0104e10 2024-09-11T00:20:21.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.909+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca01a7e50 con 0x7f0ca0104e10 2024-09-11T00:20:21.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.909+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca01060c0 con 0x7f0ca0104e10 2024-09-11T00:20:21.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.912+0000 7f0c9e000640 1 -- 172.21.15.103:0/3571460756 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f0c90010430 con 0x7f0ca0104e10 2024-09-11T00:20:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.913+0000 7f0c9e000640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0c840b5390 0x7f0c840b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.913+0000 7f0c9e000640 1 -- 172.21.15.103:0/3571460756 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f0c900e5fd0 con 0x7f0ca0104e10 2024-09-11T00:20:21.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.913+0000 7f0c97e00640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0c840b5390 0x7f0c840b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:21.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.914+0000 7f0c97e00640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0c840b5390 0x7f0c840b7850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005ab0 tx=0x7f0c88005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:21.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:21.915+0000 7f0c9e000640 1 -- 172.21.15.103:0/3571460756 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0c900acc40 con 0x7f0ca0104e10 2024-09-11T00:20:22.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:22.103+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0ca01051f0 con 0x7f0ca0104e10 2024-09-11T00:20:22.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:22.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:22.105 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:21 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:22.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:22.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:21 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:22.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:20:22.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:21 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:20:22.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:20:23.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.017+0000 7f0c9e000640 1 -- 172.21.15.103:0/3571460756 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f0ca01051f0 con 0x7f0ca0104e10 2024-09-11T00:20:23.019 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' created 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0c840b5390 msgr2=0x7f0c840b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0c840b5390 0x7f0c840b7850 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005ab0 tx=0x7f0c88005a20 comp rx=0 tx=0).stop 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca0104e10 msgr2=0x7f0ca019b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca0104e10 0x7f0ca019b220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0c9000e9f0 tx=0x7f0c9000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 shutdown_connections 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0c840b5390 0x7f0c840b7850 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ca010b1a0 0x7f0ca01a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0ca010a820 0x7f0ca019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 --2- 172.21.15.103:0/3571460756 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0ca0104e10 0x7f0ca019b220 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 >> 172.21.15.103:0/3571460756 conn(0x7f0ca0100840 msgr2=0x7f0ca0102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 shutdown_connections 2024-09-11T00:20:23.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:23.021+0000 7f0ca6200640 1 -- 172.21.15.103:0/3571460756 wait complete. 2024-09-11T00:20:23.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:23 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3571460756' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:20:23.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:23 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:20:23.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:23 smithi183 ceph-mon[35934]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:23.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:23 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3571460756' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:20:23.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:23 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:20:23.413 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:23 smithi103 ceph-mon[30578]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:23 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3571460756' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:20:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:23 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-11T00:20:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:23 smithi117 ceph-mon[35525]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:24.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-11T00:20:24.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:24 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:20:24.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:24 smithi183 ceph-mon[35934]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:20:24.417 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:24.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:24 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:20:24.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:24 smithi103 ceph-mon[30578]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:20:24.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:24 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T00:20:24.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:24 smithi117 ceph-mon[35525]: osdmap e72: 12 total, 12 up, 12 in 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.011+0000 7fd890800640 1 -- 172.21.15.103:0/1892856612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888077ff0 msgr2=0x7fd888113440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.011+0000 7fd890800640 1 --2- 172.21.15.103:0/1892856612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888077ff0 0x7fd888113440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd878009900 tx=0x7fd87802a410 comp rx=0 tx=0).stop 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.012+0000 7fd890800640 1 -- 172.21.15.103:0/1892856612 shutdown_connections 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.012+0000 7fd890800640 1 --2- 172.21.15.103:0/1892856612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd888113980 0x7fd888115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.012+0000 7fd890800640 1 --2- 172.21.15.103:0/1892856612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888077ff0 0x7fd888113440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.012+0000 7fd890800640 1 --2- 172.21.15.103:0/1892856612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd8880776d0 0x7fd888077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.012+0000 7fd890800640 1 -- 172.21.15.103:0/1892856612 >> 172.21.15.103:0/1892856612 conn(0x7fd888100a00 msgr2=0x7fd888102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.012+0000 7fd890800640 1 -- 172.21.15.103:0/1892856612 shutdown_connections 2024-09-11T00:20:25.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.012+0000 7fd890800640 1 -- 172.21.15.103:0/1892856612 wait complete. 2024-09-11T00:20:25.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 Processor -- start 2024-09-11T00:20:25.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 -- start start 2024-09-11T00:20:25.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8880776d0 0x7fd88819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888077ff0 0x7fd88819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888113980 0x7fd8881a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd888119360 con 0x7fd888113980 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881191e0 con 0x7fd888077ff0 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd890800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881194e0 con 0x7fd8880776d0 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd88de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8880776d0 0x7fd88819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd88de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8880776d0 0x7fd88819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57958/0 (socket says 172.21.15.103:57958) 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd88de00640 1 -- 172.21.15.103:0/3962492089 learned_addr learned my addr 172.21.15.103:0/3962492089 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:25.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.013+0000 7fd88e800640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888113980 0x7fd8881a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd88d400640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888077ff0 0x7fd88819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd88de00640 1 -- 172.21.15.103:0/3962492089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888077ff0 msgr2=0x7fd88819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd88de00640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888077ff0 0x7fd88819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd88de00640 1 -- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888113980 msgr2=0x7fd8881a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd88de00640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888113980 0x7fd8881a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd88de00640 1 -- 172.21.15.103:0/3962492089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8881a5910 con 0x7fd8880776d0 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd88de00640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8880776d0 0x7fd88819f860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9f0 tx=0x7fd87c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd882a00640 1 -- 172.21.15.103:0/3962492089 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00cd80 con 0x7fd8880776d0 2024-09-11T00:20:25.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.014+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8881a5c00 con 0x7fd8880776d0 2024-09-11T00:20:25.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.015+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881ac490 con 0x7fd8880776d0 2024-09-11T00:20:25.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.015+0000 7fd882a00640 1 -- 172.21.15.103:0/3962492089 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd87c0108e0 con 0x7fd8880776d0 2024-09-11T00:20:25.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.015+0000 7fd882a00640 1 -- 172.21.15.103:0/3962492089 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c018980 con 0x7fd8880776d0 2024-09-11T00:20:25.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.016+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd86c0052c0 con 0x7fd8880776d0 2024-09-11T00:20:25.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.018+0000 7fd882a00640 1 -- 172.21.15.103:0/3962492089 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fd87c010430 con 0x7fd8880776d0 2024-09-11T00:20:25.021 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.020+0000 7fd882a00640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd8700b5360 0x7fd8700b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:25.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.020+0000 7fd882a00640 1 -- 172.21.15.103:0/3962492089 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7497+0+0 (secure 0 0 0) 0x7fd87c0e5990 con 0x7fd8880776d0 2024-09-11T00:20:25.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.020+0000 7fd88d400640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd8700b5360 0x7fd8700b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:25.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.021+0000 7fd88d400640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd8700b5360 0x7fd8700b7820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd878031440 tx=0x7fd878005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:25.025 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.023+0000 7fd882a00640 1 -- 172.21.15.103:0/3962492089 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd87c0ac490 con 0x7fd8880776d0 2024-09-11T00:20:25.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:25 smithi103 ceph-mon[30578]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:20:25.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:25 smithi103 ceph-mon[30578]: pgmap v302: 33 pgs: 19 creating+peering, 13 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:25.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:25.213+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fd86c0055b0 con 0x7fd8880776d0 2024-09-11T00:20:25.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:25 smithi183 ceph-mon[35934]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:20:25.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:25 smithi183 ceph-mon[35934]: pgmap v302: 33 pgs: 19 creating+peering, 13 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:25 smithi117 ceph-mon[35525]: osdmap e73: 12 total, 12 up, 12 in 2024-09-11T00:20:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:25 smithi117 ceph-mon[35525]: pgmap v302: 33 pgs: 19 creating+peering, 13 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:26.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.038+0000 7fd882a00640 1 -- 172.21.15.103:0/3962492089 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7fd87c0b0ec0 con 0x7fd8880776d0 2024-09-11T00:20:26.042 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' created 2024-09-11T00:20:26.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd8700b5360 msgr2=0x7fd8700b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd8700b5360 0x7fd8700b7820 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd878031440 tx=0x7fd878005b50 comp rx=0 tx=0).stop 2024-09-11T00:20:26.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8880776d0 msgr2=0x7fd88819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8880776d0 0x7fd88819f860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9f0 tx=0x7fd87c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 shutdown_connections 2024-09-11T00:20:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd8700b5360 0x7fd8700b7820 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd888113980 0x7fd8881a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888077ff0 0x7fd88819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 --2- 172.21.15.103:0/3962492089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8880776d0 0x7fd88819f860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:26.049 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.046+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 >> 172.21.15.103:0/3962492089 conn(0x7fd888100a00 msgr2=0x7fd888102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:26.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.047+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 shutdown_connections 2024-09-11T00:20:26.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:26.047+0000 7fd890800640 1 -- 172.21.15.103:0/3962492089 wait complete. 2024-09-11T00:20:26.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:26 smithi183 ceph-mon[35934]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:20:26.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:26 smithi183 ceph-mon[35934]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:20:26.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:26 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3962492089' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:20:26.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:26 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:20:26.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:26 smithi103 ceph-mon[30578]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:20:26.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:26 smithi103 ceph-mon[30578]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:20:26.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:26 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3962492089' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:20:26.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:26 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:20:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:26 smithi117 ceph-mon[35525]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:20:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:26 smithi117 ceph-mon[35525]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T00:20:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:26 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3962492089' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:20:26.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:26 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-11T00:20:27.018 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T00:20:27.409 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:27 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T00:20:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:27 smithi103 ceph-mon[30578]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:20:27.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:27 smithi103 ceph-mon[30578]: pgmap v305: 97 pgs: 19 creating+peering, 74 unknown, 4 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:27.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:27 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T00:20:27.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:27 smithi117 ceph-mon[35525]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:20:27.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:27 smithi117 ceph-mon[35525]: pgmap v305: 97 pgs: 19 creating+peering, 74 unknown, 4 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:27.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:27 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-11T00:20:27.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:27 smithi183 ceph-mon[35934]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T00:20:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:27 smithi183 ceph-mon[35934]: pgmap v305: 97 pgs: 19 creating+peering, 74 unknown, 4 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.990+0000 7fddefa00640 1 -- 172.21.15.103:0/3814709393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 msgr2=0x7fdde8104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.990+0000 7fddefa00640 1 --2- 172.21.15.103:0/3814709393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde8104070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009900 tx=0x7fddd802a410 comp rx=0 tx=0).stop 2024-09-11T00:20:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.991+0000 7fddefa00640 1 -- 172.21.15.103:0/3814709393 shutdown_connections 2024-09-11T00:20:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.991+0000 7fddefa00640 1 --2- 172.21.15.103:0/3814709393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde810d0f0 0x7fdde810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.991+0000 7fddefa00640 1 --2- 172.21.15.103:0/3814709393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde81045b0 0x7fdde810caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.991+0000 7fddefa00640 1 --2- 172.21.15.103:0/3814709393 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde8104070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:28.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.991+0000 7fddefa00640 1 -- 172.21.15.103:0/3814709393 >> 172.21.15.103:0/3814709393 conn(0x7fdde80fd710 msgr2=0x7fdde80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.991+0000 7fddefa00640 1 -- 172.21.15.103:0/3814709393 shutdown_connections 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.991+0000 7fddefa00640 1 -- 172.21.15.103:0/3814709393 wait complete. 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 Processor -- start 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 -- start start 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde81045b0 0x7fdde819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde810d0f0 0x7fdde81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8112d60 con 0x7fdde8103c90 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8112be0 con 0x7fdde810d0f0 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddefa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8112ee0 con 0x7fdde81045b0 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fdded000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:28.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fdded000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54246/0 (socket says 172.21.15.103:54246) 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fdded000640 1 -- 172.21.15.103:0/3578837530 learned_addr learned my addr 172.21.15.103:0/3578837530 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fdddfe00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde81045b0 0x7fdde819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.992+0000 7fddeda00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde810d0f0 0x7fdde81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdded000640 1 -- 172.21.15.103:0/3578837530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde81045b0 msgr2=0x7fdde819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdded000640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde81045b0 0x7fdde819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdded000640 1 -- 172.21.15.103:0/3578837530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde810d0f0 msgr2=0x7fdde81a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdded000640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde810d0f0 0x7fdde81a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdded000640 1 -- 172.21.15.103:0/3578837530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde81a15a0 con 0x7fdde8103c90 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdded000640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde819b4f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005940 tx=0x7fddd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdde6000640 1 -- 172.21.15.103:0/3578837530 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd803f030 con 0x7fdde8103c90 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdde6000640 1 -- 172.21.15.103:0/3578837530 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd8003aa0 con 0x7fdde8103c90 2024-09-11T00:20:28.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde81a1830 con 0x7fdde8103c90 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fdde6000640 1 -- 172.21.15.103:0/3578837530 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8035070 con 0x7fdde8103c90 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.993+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde81a7f50 con 0x7fdde8103c90 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.995+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde8104070 con 0x7fdde8103c90 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.997+0000 7fdde6000640 1 -- 172.21.15.103:0/3578837530 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fddd803bca0 con 0x7fdde8103c90 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.998+0000 7fdde6000640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fddcc0b52e0 0x7fddcc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.999+0000 7fdde6000640 1 -- 172.21.15.103:0/3578837530 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7879+0+0 (secure 0 0 0) 0x7fddd8106620 con 0x7fdde8103c90 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.999+0000 7fdddfe00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fddcc0b52e0 0x7fddcc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:27.999+0000 7fdddfe00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fddcc0b52e0 0x7fddcc0b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdde810e850 tx=0x7fddd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:28.034 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:28.001+0000 7fdde6000640 1 -- 172.21.15.103:0/3578837530 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fddd80ccfa0 con 0x7fdde8103c90 2024-09-11T00:20:28.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:28 smithi103 ceph-mon[30578]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:20:28.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:28.227+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fdde81a81b0 con 0x7fdde8103c90 2024-09-11T00:20:28.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:28 smithi117 ceph-mon[35525]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:20:28.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:28 smithi183 ceph-mon[35934]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T00:20:29.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.052+0000 7fdde6000640 1 -- 172.21.15.103:0/3578837530 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fddd80d19d0 con 0x7fdde8103c90 2024-09-11T00:20:29.054 INFO:teuthology.orchestra.run.smithi103.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T00:20:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fddcc0b52e0 msgr2=0x7fddcc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fddcc0b52e0 0x7fddcc0b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdde810e850 tx=0x7fddd0005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 msgr2=0x7fdde819b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:29.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde819b4f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddd8005940 tx=0x7fddd8004270 comp rx=0 tx=0).stop 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 shutdown_connections 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fddcc0b52e0 0x7fddcc0b77a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdde810d0f0 0x7fdde81a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdde81045b0 0x7fdde819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 --2- 172.21.15.103:0/3578837530 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdde8103c90 0x7fdde819b4f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.055+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 >> 172.21.15.103:0/3578837530 conn(0x7fdde80fd710 msgr2=0x7fdde810f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.056+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 shutdown_connections 2024-09-11T00:20:29.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:29.056+0000 7fddefa00640 1 -- 172.21.15.103:0/3578837530 wait complete. 2024-09-11T00:20:29.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:29 smithi103 ceph-mon[30578]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:20:29.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:29 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3578837530' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:20:29.410 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:29 smithi103 ceph-mon[30578]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:20:29.411 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:29 smithi103 ceph-mon[30578]: pgmap v308: 97 pgs: 25 creating+peering, 44 unknown, 28 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:29.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:29 smithi117 ceph-mon[35525]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:20:29.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:29 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3578837530' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:20:29.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:29 smithi117 ceph-mon[35525]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:20:29.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:29 smithi117 ceph-mon[35525]: pgmap v308: 97 pgs: 25 creating+peering, 44 unknown, 28 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:29 smithi183 ceph-mon[35934]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T00:20:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:29 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3578837530' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-11T00:20:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:29 smithi183 ceph-mon[35934]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T00:20:29.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:29 smithi183 ceph-mon[35934]: pgmap v308: 97 pgs: 25 creating+peering, 44 unknown, 28 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:29.950 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-11T00:20:29.950 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-11T00:20:30.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:30 smithi103 ceph-mon[30578]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T00:20:30.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:30 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3578837530' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:20:30.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:30 smithi103 ceph-mon[30578]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:20:30.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:30 smithi103 ceph-mon[30578]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:20:30.340 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:30.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:30 smithi117 ceph-mon[35525]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T00:20:30.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:30 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3578837530' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:20:30.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:30 smithi117 ceph-mon[35525]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:20:30.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:30 smithi117 ceph-mon[35525]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:20:30.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:30 smithi183 ceph-mon[35934]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T00:20:30.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:30 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3578837530' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T00:20:30.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:30 smithi183 ceph-mon[35934]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T00:20:30.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:30 smithi183 ceph-mon[35934]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T00:20:30.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.956+0000 7f51e6a00640 1 -- 172.21.15.103:0/1271667200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0102c10 msgr2=0x7f51e0102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:30.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.956+0000 7f51e6a00640 1 --2- 172.21.15.103:0/1271667200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0102c10 0x7f51e0102ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009ca0 tx=0x7f51d402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:20:30.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.957+0000 7f51e6a00640 1 -- 172.21.15.103:0/1271667200 shutdown_connections 2024-09-11T00:20:30.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.957+0000 7f51e6a00640 1 --2- 172.21.15.103:0/1271667200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0108fa0 0x7f51e010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:30.959 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.957+0000 7f51e6a00640 1 --2- 172.21.15.103:0/1271667200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e0108620 0x7f51e0108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:30.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.957+0000 7f51e6a00640 1 --2- 172.21.15.103:0/1271667200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0102c10 0x7f51e0102ff0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:30.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.957+0000 7f51e6a00640 1 -- 172.21.15.103:0/1271667200 >> 172.21.15.103:0/1271667200 conn(0x7f51e00fe6c0 msgr2=0x7f51e0100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:30.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.957+0000 7f51e6a00640 1 -- 172.21.15.103:0/1271667200 shutdown_connections 2024-09-11T00:20:30.960 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.957+0000 7f51e6a00640 1 -- 172.21.15.103:0/1271667200 wait complete. 2024-09-11T00:20:30.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 Processor -- start 2024-09-11T00:20:30.961 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 -- start start 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102c10 0x7f51e019b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0108620 0x7f51e019b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e0108fa0 0x7f51e01a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f51e0112c60 con 0x7f51e0108fa0 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f51e0112ae0 con 0x7f51e0102c10 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.958+0000 7f51e6a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51e0112de0 con 0x7f51e0108620 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102c10 0x7f51e019b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102c10 0x7f51e019b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:37366/0 (socket says 172.21.15.103:37366) 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 -- 172.21.15.103:0/244898601 learned_addr learned my addr 172.21.15.103:0/244898601 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e4c00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0108620 0x7f51e019b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e6000640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e0108fa0 0x7f51e01a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 -- 172.21.15.103:0/244898601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0108620 msgr2=0x7f51e019b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:30.962 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0108620 0x7f51e019b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 -- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e0108fa0 msgr2=0x7f51e01a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e0108fa0 0x7f51e01a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 -- 172.21.15.103:0/244898601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e01a12b0 con 0x7f51e0102c10 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.959+0000 7f51e5600640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102c10 0x7f51e019b200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51d4033040 tx=0x7f51d40316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.960+0000 7f51de000640 1 -- 172.21.15.103:0/244898601 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d4003b70 con 0x7f51e0102c10 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.960+0000 7f51de000640 1 -- 172.21.15.103:0/244898601 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51d4003d10 con 0x7f51e0102c10 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.960+0000 7f51de000640 1 -- 172.21.15.103:0/244898601 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d402f9e0 con 0x7f51e0102c10 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.960+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e01a1540 con 0x7f51e0102c10 2024-09-11T00:20:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.960+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e01a7c60 con 0x7f51e0102c10 2024-09-11T00:20:30.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.961+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e0102ff0 con 0x7f51e0102c10 2024-09-11T00:20:30.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.963+0000 7f51de000640 1 -- 172.21.15.103:0/244898601 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f51d402fb80 con 0x7f51e0102c10 2024-09-11T00:20:30.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.964+0000 7f51de000640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51c80b5410 0x7f51c80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:30.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.965+0000 7f51de000640 1 -- 172.21.15.103:0/244898601 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7f51d4106de0 con 0x7f51e0102c10 2024-09-11T00:20:30.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.965+0000 7f51e4c00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51c80b5410 0x7f51c80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:30.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.965+0000 7f51e4c00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51c80b5410 0x7f51c80b78d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51e0107fa0 tx=0x7f51cc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:30.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:30.966+0000 7f51de000640 1 -- 172.21.15.103:0/244898601 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f51d40cdd50 con 0x7f51e0102c10 2024-09-11T00:20:31.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:31.161+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f51e002e5c0 con 0x7f51e0102c10 2024-09-11T00:20:31.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:31 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] up:active 2024-09-11T00:20:31.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:31 smithi103 ceph-mon[30578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:31.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:31 smithi103 ceph-mon[30578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:31.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:31 smithi103 ceph-mon[30578]: pgmap v310: 97 pgs: 6 creating+peering, 14 unknown, 77 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:31.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:31 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] up:active 2024-09-11T00:20:31.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:31 smithi117 ceph-mon[35525]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:31.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:31 smithi117 ceph-mon[35525]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:31.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:31 smithi117 ceph-mon[35525]: pgmap v310: 97 pgs: 6 creating+peering, 14 unknown, 77 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:31.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:31 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] up:active 2024-09-11T00:20:31.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:31 smithi183 ceph-mon[35934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:31.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:31 smithi183 ceph-mon[35934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:31.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:31 smithi183 ceph-mon[35934]: pgmap v310: 97 pgs: 6 creating+peering, 14 unknown, 77 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:32.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.074+0000 7f51de000640 1 -- 172.21.15.103:0/244898601 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7f51d40cd650 con 0x7f51e0102c10 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.077+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51c80b5410 msgr2=0x7f51c80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.077+0000 7f51e6a00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51c80b5410 0x7f51c80b78d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51e0107fa0 tx=0x7f51cc005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.077+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102c10 msgr2=0x7f51e019b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.077+0000 7f51e6a00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102c10 0x7f51e019b200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51d4033040 tx=0x7f51d40316f0 comp rx=0 tx=0).stop 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 shutdown_connections 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f51c80b5410 0x7f51c80b78d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51e0108fa0 0x7f51e01a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51e0108620 0x7f51e019b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 --2- 172.21.15.103:0/244898601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51e0102c10 0x7f51e019b200 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 >> 172.21.15.103:0/244898601 conn(0x7f51e00fe6c0 msgr2=0x7f51e0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 shutdown_connections 2024-09-11T00:20:32.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:32.078+0000 7f51e6a00640 1 -- 172.21.15.103:0/244898601 wait complete. 2024-09-11T00:20:32.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:32 smithi103 ceph-mon[30578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:20:32.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:32 smithi103 ceph-mon[30578]: Cluster is now healthy 2024-09-11T00:20:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:32 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/244898601' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:20:32.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:32 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:20:32.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:32 smithi117 ceph-mon[35525]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:20:32.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:32 smithi117 ceph-mon[35525]: Cluster is now healthy 2024-09-11T00:20:32.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:32 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/244898601' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:20:32.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:32 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:20:32.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:32 smithi183 ceph-mon[35934]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T00:20:32.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:32 smithi183 ceph-mon[35934]: Cluster is now healthy 2024-09-11T00:20:32.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:32 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/244898601' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:20:32.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:32 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-11T00:20:33.082 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T00:20:33.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:33 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T00:20:33.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:33 smithi103 ceph-mon[30578]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T00:20:33.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:33 smithi103 ceph-mon[30578]: pgmap v312: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 192 B/s wr, 1 op/s 2024-09-11T00:20:33.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:33 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T00:20:33.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:33 smithi117 ceph-mon[35525]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T00:20:33.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:33 smithi117 ceph-mon[35525]: pgmap v312: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 192 B/s wr, 1 op/s 2024-09-11T00:20:33.456 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:33.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:33 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-11T00:20:33.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:33 smithi183 ceph-mon[35934]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T00:20:33.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:33 smithi183 ceph-mon[35934]: pgmap v312: 97 pgs: 97 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 192 B/s wr, 1 op/s 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 -- 172.21.15.103:0/1350590614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f533010a9a0 msgr2=0x7f533010ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 --2- 172.21.15.103:0/1350590614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f533010a9a0 0x7f533010ade0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f531c009900 tx=0x7f531c02a400 comp rx=0 tx=0).stop 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 -- 172.21.15.103:0/1350590614 shutdown_connections 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 --2- 172.21.15.103:0/1350590614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f533010b320 0x7f5330111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 --2- 172.21.15.103:0/1350590614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f533010a9a0 0x7f533010ade0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 --2- 172.21.15.103:0/1350590614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5330104f90 0x7f5330105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 -- 172.21.15.103:0/1350590614 >> 172.21.15.103:0/1350590614 conn(0x7f5330100a00 msgr2=0x7f5330102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 -- 172.21.15.103:0/1350590614 shutdown_connections 2024-09-11T00:20:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.202+0000 7f5335c00640 1 -- 172.21.15.103:0/1350590614 wait complete. 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.203+0000 7f5335c00640 1 Processor -- start 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.203+0000 7f5335c00640 1 -- start start 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f5335c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5330104f90 0x7f533019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f5335c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533010a9a0 0x7f533019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f5335c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f533010b320 0x7f53301a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f5335c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5330114f90 con 0x7f533010b320 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f5335c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5330114e10 con 0x7f5330104f90 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f5335c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5330115110 con 0x7f533010a9a0 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f532ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5330104f90 0x7f533019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f532e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533010a9a0 0x7f533019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:34.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f532ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5330104f90 0x7f533019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:47432/0 (socket says 172.21.15.103:47432) 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f532ea00640 1 -- 172.21.15.103:0/3818745163 learned_addr learned my addr 172.21.15.103:0/3818745163 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.204+0000 7f532f400640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f533010b320 0x7f53301a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f532ea00640 1 -- 172.21.15.103:0/3818745163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533010a9a0 msgr2=0x7f533019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f532ea00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533010a9a0 0x7f533019ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f532ea00640 1 -- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f533010b320 msgr2=0x7f53301a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f532ea00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f533010b320 0x7f53301a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f532ea00640 1 -- 172.21.15.103:0/3818745163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53301a1590 con 0x7f5330104f90 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f532e000640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533010a9a0 0x7f533019ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:34.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f532ea00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5330104f90 0x7f533019b4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f532000e9f0 tx=0x7f532000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f5327400640 1 -- 172.21.15.103:0/3818745163 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532000ccc0 con 0x7f5330104f90 2024-09-11T00:20:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f5327400640 1 -- 172.21.15.103:0/3818745163 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f532000ce60 con 0x7f5330104f90 2024-09-11T00:20:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53301a1880 con 0x7f5330104f90 2024-09-11T00:20:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f5327400640 1 -- 172.21.15.103:0/3818745163 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320010430 con 0x7f5330104f90 2024-09-11T00:20:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.205+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53301a7f40 con 0x7f5330104f90 2024-09-11T00:20:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.206+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5330105370 con 0x7f5330104f90 2024-09-11T00:20:34.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.209+0000 7f5327400640 1 -- 172.21.15.103:0/3818745163 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f53200046c0 con 0x7f5330104f90 2024-09-11T00:20:34.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.210+0000 7f5327400640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f53140b52e0 0x7f53140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:34.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.210+0000 7f5327400640 1 -- 172.21.15.103:0/3818745163 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f53200e6230 con 0x7f5330104f90 2024-09-11T00:20:34.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.211+0000 7f532e000640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f53140b52e0 0x7f53140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:34.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.211+0000 7f532e000640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f53140b52e0 0x7f53140b77a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f531c005ab0 tx=0x7f531c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:34.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.212+0000 7f5327400640 1 -- 172.21.15.103:0/3818745163 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5320056220 con 0x7f5330104f90 2024-09-11T00:20:34.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:34.412+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f533019c610 con 0x7f5330104f90 2024-09-11T00:20:34.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:20:34 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: 2024-09-11T00:20:34.139+0000 7f56541e2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T00:20:34.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:34 smithi103 ceph-mon[30578]: pgmap v313: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-11T00:20:34.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:34 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3818745163' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:20:34.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:34 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:20:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:34 smithi183 ceph-mon[35934]: pgmap v313: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-11T00:20:34.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:34 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3818745163' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:20:34.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:34 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:20:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:34 smithi117 ceph-mon[35525]: pgmap v313: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-11T00:20:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:34 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3818745163' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:20:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:34 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-11T00:20:36.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.429+0000 7f5327400640 1 -- 172.21.15.103:0/3818745163 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f533019c610 con 0x7f5330104f90 2024-09-11T00:20:36.435 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data_ec' created 2024-09-11T00:20:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f53140b52e0 msgr2=0x7f53140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f53140b52e0 0x7f53140b77a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f531c005ab0 tx=0x7f531c005a20 comp rx=0 tx=0).stop 2024-09-11T00:20:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5330104f90 msgr2=0x7f533019b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5330104f90 0x7f533019b4e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f532000e9f0 tx=0x7f532000edf0 comp rx=0 tx=0).stop 2024-09-11T00:20:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 shutdown_connections 2024-09-11T00:20:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f53140b52e0 0x7f53140b77a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f533010b320 0x7f53301a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f533010a9a0 0x7f533019ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 --2- 172.21.15.103:0/3818745163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5330104f90 0x7f533019b4e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 >> 172.21.15.103:0/3818745163 conn(0x7f5330100a00 msgr2=0x7f5330102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 shutdown_connections 2024-09-11T00:20:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:36.436+0000 7f5335c00640 1 -- 172.21.15.103:0/3818745163 wait complete. 2024-09-11T00:20:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:36 smithi117 ceph-mon[35525]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T00:20:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:36 smithi117 ceph-mon[35525]: pgmap v315: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T00:20:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:36 smithi183 ceph-mon[35934]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T00:20:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:36 smithi183 ceph-mon[35934]: pgmap v315: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T00:20:36.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:36 smithi103 ceph-mon[30578]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T00:20:36.832 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:36 smithi103 ceph-mon[30578]: pgmap v315: 97 pgs: 97 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T00:20:37.412 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-11T00:20:37.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:37 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:20:37.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:37 smithi103 ceph-mon[30578]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T00:20:37.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:37 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:20:37.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:37 smithi117 ceph-mon[35525]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T00:20:37.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:37 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T00:20:37.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:37 smithi183 ceph-mon[35934]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T00:20:37.806 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:38.420 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 -- 172.21.15.103:0/3486467274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576410a9b0 msgr2=0x7f576410adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 --2- 172.21.15.103:0/3486467274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576410a9b0 0x7f576410adf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5750009860 tx=0x7f575002a340 comp rx=0 tx=0).stop 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 -- 172.21.15.103:0/3486467274 shutdown_connections 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 --2- 172.21.15.103:0/3486467274 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410b330 0x7f5764111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 --2- 172.21.15.103:0/3486467274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576410a9b0 0x7f576410adf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 --2- 172.21.15.103:0/3486467274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5764104fa0 0x7f5764105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 -- 172.21.15.103:0/3486467274 >> 172.21.15.103:0/3486467274 conn(0x7f5764100a50 msgr2=0x7f5764102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 -- 172.21.15.103:0/3486467274 shutdown_connections 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.383+0000 7f5769200640 1 -- 172.21.15.103:0/3486467274 wait complete. 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 Processor -- start 2024-09-11T00:20:38.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 -- start start 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5764104fa0 0x7f576419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410a9b0 0x7f576419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f576410b330 0x7f57641a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5764114f50 con 0x7f576410a9b0 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5764114dd0 con 0x7f5764104fa0 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.384+0000 7f5769200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f57641150d0 con 0x7f576410b330 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410a9b0 0x7f576419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5764104fa0 0x7f576419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410a9b0 0x7f576419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59710/0 (socket says 172.21.15.103:59710) 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761400640 1 -- 172.21.15.103:0/1944348622 learned_addr learned my addr 172.21.15.103:0/1944348622 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5764104fa0 0x7f576419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:47448/0 (socket says 172.21.15.103:47448) 2024-09-11T00:20:38.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5762800640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f576410b330 0x7f57641a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761e00640 1 -- 172.21.15.103:0/1944348622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f576410b330 msgr2=0x7f57641a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761e00640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f576410b330 0x7f57641a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761e00640 1 -- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410a9b0 msgr2=0x7f576419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761e00640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410a9b0 0x7f576419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761e00640 1 -- 172.21.15.103:0/1944348622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57641a1570 con 0x7f5764104fa0 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5761400640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410a9b0 0x7f576419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.385+0000 7f5762800640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f576410b330 0x7f57641a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.386+0000 7f5761e00640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5764104fa0 0x7f576419b4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5754009fd0 tx=0x7f575400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.386+0000 7f575aa00640 1 -- 172.21.15.103:0/1944348622 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754004b20 con 0x7f5764104fa0 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.386+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57641a1860 con 0x7f5764104fa0 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.386+0000 7f575aa00640 1 -- 172.21.15.103:0/1944348622 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5754004cc0 con 0x7f5764104fa0 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.386+0000 7f575aa00640 1 -- 172.21.15.103:0/1944348622 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754010720 con 0x7f5764104fa0 2024-09-11T00:20:38.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.386+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57641a7f20 con 0x7f5764104fa0 2024-09-11T00:20:38.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.387+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5764105380 con 0x7f5764104fa0 2024-09-11T00:20:38.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.389+0000 7f575aa00640 1 -- 172.21.15.103:0/1944348622 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f57540108c0 con 0x7f5764104fa0 2024-09-11T00:20:38.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.391+0000 7f575aa00640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57480b52e0 0x7f57480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:38.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.391+0000 7f575aa00640 1 -- 172.21.15.103:0/1944348622 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8770+0+0 (secure 0 0 0) 0x7f57540e6550 con 0x7f5764104fa0 2024-09-11T00:20:38.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.391+0000 7f5761400640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57480b52e0 0x7f57480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:38.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.392+0000 7f5761400640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57480b52e0 0x7f57480b77a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5764106fb0 tx=0x7f5750030340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:38.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.393+0000 7f575aa00640 1 -- 172.21.15.103:0/1944348622 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f57540ad190 con 0x7f5764104fa0 2024-09-11T00:20:38.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:38.584+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f576402e5c0 con 0x7f5764104fa0 2024-09-11T00:20:38.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:38 smithi103 ceph-mon[30578]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T00:20:38.586 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:38 smithi103 ceph-mon[30578]: pgmap v318: 129 pgs: 9 creating+peering, 22 unknown, 98 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-11T00:20:38.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:38 smithi183 ceph-mon[35934]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T00:20:38.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:38 smithi183 ceph-mon[35934]: pgmap v318: 129 pgs: 9 creating+peering, 22 unknown, 98 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-11T00:20:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:38 smithi117 ceph-mon[35525]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T00:20:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:38 smithi117 ceph-mon[35525]: pgmap v318: 129 pgs: 9 creating+peering, 22 unknown, 98 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-11T00:20:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.447+0000 7f575aa00640 1 -- 172.21.15.103:0/1944348622 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f57540aca90 con 0x7f5764104fa0 2024-09-11T00:20:39.449 INFO:teuthology.orchestra.run.smithi103.stderr:set pool 4 allow_ec_overwrites to true 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57480b52e0 msgr2=0x7f57480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57480b52e0 0x7f57480b77a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5764106fb0 tx=0x7f5750030340 comp rx=0 tx=0).stop 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5764104fa0 msgr2=0x7f576419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5764104fa0 0x7f576419b4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5754009fd0 tx=0x7f575400ef90 comp rx=0 tx=0).stop 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 shutdown_connections 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57480b52e0 0x7f57480b77a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f576410b330 0x7f57641a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f576410a9b0 0x7f576419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 --2- 172.21.15.103:0/1944348622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5764104fa0 0x7f576419b4c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 >> 172.21.15.103:0/1944348622 conn(0x7f5764100a50 msgr2=0x7f5764102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:39.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 shutdown_connections 2024-09-11T00:20:39.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:39.450+0000 7f5769200640 1 -- 172.21.15.103:0/1944348622 wait complete. 2024-09-11T00:20:39.732 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:39 smithi103 ceph-mon[30578]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:20:39.732 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:39 smithi103 ceph-mon[30578]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T00:20:39.733 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:39 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1944348622' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:20:39.733 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:39 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:20:39.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:39 smithi183 ceph-mon[35934]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:20:39.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:39 smithi183 ceph-mon[35934]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T00:20:39.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:39 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1944348622' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:20:39.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:39 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:20:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:39 smithi117 ceph-mon[35525]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T00:20:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:39 smithi117 ceph-mon[35525]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T00:20:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:39 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1944348622' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:20:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:39 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-11T00:20:40.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-11T00:20:40.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:40 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T00:20:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:40 smithi103 ceph-mon[30578]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T00:20:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:40 smithi103 ceph-mon[30578]: pgmap v321: 129 pgs: 9 creating+peering, 5 unknown, 115 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:40.681 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:40.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:40 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T00:20:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:40 smithi183 ceph-mon[35934]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T00:20:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:40 smithi183 ceph-mon[35934]: pgmap v321: 129 pgs: 9 creating+peering, 5 unknown, 115 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:40 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-11T00:20:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:40 smithi117 ceph-mon[35525]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T00:20:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:40 smithi117 ceph-mon[35525]: pgmap v321: 129 pgs: 9 creating+peering, 5 unknown, 115 active+clean; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:41.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.313+0000 7f4279c00640 1 -- 172.21.15.103:0/675589309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274108620 msgr2=0x7f4274108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:41.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.313+0000 7f4279c00640 1 --2- 172.21.15.103:0/675589309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274108620 0x7f4274108a60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4260009900 tx=0x7f426002a410 comp rx=0 tx=0).stop 2024-09-11T00:20:41.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.313+0000 7f4279c00640 1 -- 172.21.15.103:0/675589309 shutdown_connections 2024-09-11T00:20:41.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.313+0000 7f4279c00640 1 --2- 172.21.15.103:0/675589309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4274108fa0 0x7f427410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.313+0000 7f4279c00640 1 --2- 172.21.15.103:0/675589309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274108620 0x7f4274108a60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.313+0000 7f4279c00640 1 --2- 172.21.15.103:0/675589309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4274102c10 0x7f4274102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.313+0000 7f4279c00640 1 -- 172.21.15.103:0/675589309 >> 172.21.15.103:0/675589309 conn(0x7f42740fe6c0 msgr2=0x7f4274100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.314+0000 7f4279c00640 1 -- 172.21.15.103:0/675589309 shutdown_connections 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.314+0000 7f4279c00640 1 -- 172.21.15.103:0/675589309 wait complete. 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 Processor -- start 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 -- start start 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274102c10 0x7f42740754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4274108620 0x7f42740759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4274108fa0 0x7f4274075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4274112c40 con 0x7f4274102c10 2024-09-11T00:20:41.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4274112ac0 con 0x7f4274108620 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4279c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4274112dc0 con 0x7f4274108fa0 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4273e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274102c10 0x7f42740754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4273e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274102c10 0x7f42740754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59724/0 (socket says 172.21.15.103:59724) 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.315+0000 7f4273e00640 1 -- 172.21.15.103:0/973785016 learned_addr learned my addr 172.21.15.103:0/973785016 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4273400640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4274108620 0x7f42740759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4279200640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4274108fa0 0x7f4274075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4273e00640 1 -- 172.21.15.103:0/973785016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4274108fa0 msgr2=0x7f4274075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4273e00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4274108fa0 0x7f4274075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4273e00640 1 -- 172.21.15.103:0/973785016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4274108620 msgr2=0x7f42740759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4273e00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4274108620 0x7f42740759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4273e00640 1 -- 172.21.15.103:0/973785016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42741a7450 con 0x7f4274102c10 2024-09-11T00:20:41.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.316+0000 7f4273e00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274102c10 0x7f42740754a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f426400ebc0 tx=0x7f426400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.317+0000 7f4270c00640 1 -- 172.21.15.103:0/973785016 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264010630 con 0x7f4274102c10 2024-09-11T00:20:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.317+0000 7f4270c00640 1 -- 172.21.15.103:0/973785016 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4264010c80 con 0x7f4274102c10 2024-09-11T00:20:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.317+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42741a7740 con 0x7f4274102c10 2024-09-11T00:20:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.317+0000 7f4270c00640 1 -- 172.21.15.103:0/973785016 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264018b60 con 0x7f4274102c10 2024-09-11T00:20:41.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.317+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42741a7ca0 con 0x7f4274102c10 2024-09-11T00:20:41.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.318+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4274102ff0 con 0x7f4274102c10 2024-09-11T00:20:41.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.320+0000 7f4270c00640 1 -- 172.21.15.103:0/973785016 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f42640107d0 con 0x7f4274102c10 2024-09-11T00:20:41.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.322+0000 7f4270c00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f42580b5410 0x7f42580b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:41.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.322+0000 7f4270c00640 1 -- 172.21.15.103:0/973785016 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f42640e6880 con 0x7f4274102c10 2024-09-11T00:20:41.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.322+0000 7f4273400640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f42580b5410 0x7f42580b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:41.324 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.323+0000 7f4273400640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f42580b5410 0x7f42580b78d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4260005ab0 tx=0x7f4260005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:41.326 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.324+0000 7f4270c00640 1 -- 172.21.15.103:0/973785016 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f42640ace70 con 0x7f4274102c10 2024-09-11T00:20:41.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.547+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f42741a8090 con 0x7f4274102c10 2024-09-11T00:20:41.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.600+0000 7f4270c00640 1 -- 172.21.15.103:0/973785016 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f42640b18a0 con 0x7f4274102c10 2024-09-11T00:20:41.602 INFO:teuthology.orchestra.run.smithi103.stderr:added data pool 4 to fsmap 2024-09-11T00:20:41.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f42580b5410 msgr2=0x7f42580b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:41.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f42580b5410 0x7f42580b78d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4260005ab0 tx=0x7f4260005a20 comp rx=0 tx=0).stop 2024-09-11T00:20:41.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274102c10 msgr2=0x7f42740754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:41.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274102c10 0x7f42740754a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f426400ebc0 tx=0x7f426400c6a0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 shutdown_connections 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f42580b5410 0x7f42580b78d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4274108fa0 0x7f4274075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4274108620 0x7f42740759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 --2- 172.21.15.103:0/973785016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4274102c10 0x7f42740754a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.603+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 >> 172.21.15.103:0/973785016 conn(0x7f42740fe6c0 msgr2=0x7f4274100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.604+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 shutdown_connections 2024-09-11T00:20:41.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:41.604+0000 7f4279c00640 1 -- 172.21.15.103:0/973785016 wait complete. 2024-09-11T00:20:41.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:41 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/973785016' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:20:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:41 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/973785016' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:20:42.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:41 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/973785016' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-11T00:20:42.534 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:20:42.908 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:42.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:42 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/973785016' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T00:20:42.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:42 smithi103 ceph-mon[30578]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T00:20:42.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:42 smithi103 ceph-mon[30578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:42.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:42 smithi103 ceph-mon[30578]: pgmap v323: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:42.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:42 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/973785016' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T00:20:42.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:42 smithi183 ceph-mon[35934]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T00:20:42.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:42 smithi183 ceph-mon[35934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:42.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:42 smithi183 ceph-mon[35934]: pgmap v323: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:42.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:42 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/973785016' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-11T00:20:42.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:42 smithi117 ceph-mon[35525]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T00:20:42.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:42 smithi117 ceph-mon[35525]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:20:42.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:42 smithi117 ceph-mon[35525]: pgmap v323: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:43.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.596+0000 7fd73f800640 1 -- 172.21.15.103:0/2993070795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380754c0 msgr2=0x7fd7380758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.596+0000 7fd73f800640 1 --2- 172.21.15.103:0/2993070795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380754c0 0x7fd7380758a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd728009900 tx=0x7fd72802a410 comp rx=0 tx=0).stop 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.596+0000 7fd73f800640 1 -- 172.21.15.103:0/2993070795 shutdown_connections 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.596+0000 7fd73f800640 1 --2- 172.21.15.103:0/2993070795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd73810f3b0 0x7fd7381117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.596+0000 7fd73f800640 1 --2- 172.21.15.103:0/2993070795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd738075de0 0x7fd73810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.596+0000 7fd73f800640 1 --2- 172.21.15.103:0/2993070795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380754c0 0x7fd7380758a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.596+0000 7fd73f800640 1 -- 172.21.15.103:0/2993070795 >> 172.21.15.103:0/2993070795 conn(0x7fd7380fe810 msgr2=0x7fd738100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.597+0000 7fd73f800640 1 -- 172.21.15.103:0/2993070795 shutdown_connections 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.597+0000 7fd73f800640 1 -- 172.21.15.103:0/2993070795 wait complete. 2024-09-11T00:20:43.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.598+0000 7fd73f800640 1 Processor -- start 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.598+0000 7fd73f800640 1 -- start start 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73f800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7380754c0 0x7fd73819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73f800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd738075de0 0x7fd73819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73f800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73810f3b0 0x7fd7381a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73f800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd738114fa0 con 0x7fd73810f3b0 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73f800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd738114e20 con 0x7fd738075de0 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73f800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd738115120 con 0x7fd7380754c0 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7380754c0 0x7fd73819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd738075de0 0x7fd73819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:43.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7380754c0 0x7fd73819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:43832/0 (socket says 172.21.15.103:43832) 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73ce00640 1 -- 172.21.15.103:0/159857432 learned_addr learned my addr 172.21.15.103:0/159857432 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73d800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73810f3b0 0x7fd7381a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd72fe00640 1 -- 172.21.15.103:0/159857432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7380754c0 msgr2=0x7fd73819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd72fe00640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7380754c0 0x7fd73819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd72fe00640 1 -- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73810f3b0 msgr2=0x7fd7381a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd72fe00640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73810f3b0 0x7fd7381a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd72fe00640 1 -- 172.21.15.103:0/159857432 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7381a15c0 con 0x7fd738075de0 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.599+0000 7fd73d800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73810f3b0 0x7fd7381a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.600+0000 7fd72fe00640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd738075de0 0x7fd73819ba50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd72000e9f0 tx=0x7fd72000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.600+0000 7fd736000640 1 -- 172.21.15.103:0/159857432 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72000ccc0 con 0x7fd738075de0 2024-09-11T00:20:43.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.600+0000 7fd736000640 1 -- 172.21.15.103:0/159857432 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd720005020 con 0x7fd738075de0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.600+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7381a18b0 con 0x7fd738075de0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.600+0000 7fd736000640 1 -- 172.21.15.103:0/159857432 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720004a00 con 0x7fd738075de0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.600+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7381a7f70 con 0x7fd738075de0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.601+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7380758a0 con 0x7fd738075de0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.603+0000 7fd736000640 1 -- 172.21.15.103:0/159857432 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fd720004ba0 con 0x7fd738075de0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.607+0000 7fd736000640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd71c0b52e0 0x7fd71c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.607+0000 7fd736000640 1 -- 172.21.15.103:0/159857432 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fd7200e5ed0 con 0x7fd738075de0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.607+0000 7fd73ce00640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd71c0b52e0 0x7fd71c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.608+0000 7fd73ce00640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd71c0b52e0 0x7fd71c0b77a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd728031440 tx=0x7fd7280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:43.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.608+0000 7fd736000640 1 -- 172.21.15.103:0/159857432 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd7200acaf0 con 0x7fd738075de0 2024-09-11T00:20:43.839 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.837+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd738109fa0 con 0x7fd738075de0 2024-09-11T00:20:43.840 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.837+0000 7fd736000640 1 -- 172.21.15.103:0/159857432 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11860 (secure 0 0 0) 0x7fd7200ac3f0 con 0x7fd738075de0 2024-09-11T00:20:43.840 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:43.840 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"btime":"2024-09-11T00:20:41:599300+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24643,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:20:41.549050+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24656,"qdb_cluster":[24656]},"id":1}]} 2024-09-11T00:20:43.841 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 17 2024-09-11T00:20:43.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd71c0b52e0 msgr2=0x7fd71c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:43.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd71c0b52e0 0x7fd71c0b77a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd728031440 tx=0x7fd7280058d0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd738075de0 msgr2=0x7fd73819ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd738075de0 0x7fd73819ba50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd72000e9f0 tx=0x7fd72000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 shutdown_connections 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd71c0b52e0 0x7fd71c0b77a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd73810f3b0 0x7fd7381a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd738075de0 0x7fd73819ba50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 --2- 172.21.15.103:0/159857432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7380754c0 0x7fd73819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.840+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 >> 172.21.15.103:0/159857432 conn(0x7fd7380fe810 msgr2=0x7fd7380ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.841+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 shutdown_connections 2024-09-11T00:20:43.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:43.841+0000 7fd73f800640 1 -- 172.21.15.103:0/159857432 wait complete. 2024-09-11T00:20:43.856 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:43 smithi103 ceph-mon[30578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:20:43.856 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:43 smithi103 ceph-mon[30578]: Cluster is now healthy 2024-09-11T00:20:43.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:43 smithi117 ceph-mon[35525]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:20:43.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:43 smithi117 ceph-mon[35525]: Cluster is now healthy 2024-09-11T00:20:43.943 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:43 smithi183 ceph-mon[35934]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T00:20:43.943 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:43 smithi183 ceph-mon[35934]: Cluster is now healthy 2024-09-11T00:20:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:44 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/159857432' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:44 smithi103 ceph-mon[30578]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:44 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/159857432' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:44 smithi117 ceph-mon[35525]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:44.987 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:20:45.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:44 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/159857432' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:45.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:44 smithi183 ceph-mon[35934]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:45.361 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:45.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 -- 172.21.15.103:0/1667581612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc10d0f0 msgr2=0x7fd1cc10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:45.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/1667581612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc10d0f0 0x7fd1cc10f550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd1c4009ca0 tx=0x7fd1c402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:20:45.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 -- 172.21.15.103:0/1667581612 shutdown_connections 2024-09-11T00:20:45.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/1667581612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc10d0f0 0x7fd1cc10f550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:45.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/1667581612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc1045b0 0x7fd1cc10caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:45.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/1667581612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc103c90 0x7fd1cc104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:45.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 -- 172.21.15.103:0/1667581612 >> 172.21.15.103:0/1667581612 conn(0x7fd1cc0fd710 msgr2=0x7fd1cc0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 -- 172.21.15.103:0/1667581612 shutdown_connections 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.962+0000 7fd1d2e00640 1 -- 172.21.15.103:0/1667581612 wait complete. 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 Processor -- start 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 -- start start 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc103c90 0x7fd1cc101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc1045b0 0x7fd1cc101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc10d0f0 0x7fd1cc1023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc112d60 con 0x7fd1cc103c90 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc112be0 con 0x7fd1cc1045b0 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1d2e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1cc112ee0 con 0x7fd1cc10d0f0 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.963+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc103c90 0x7fd1cc101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc103c90 0x7fd1cc101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:48208/0 (socket says 172.21.15.103:48208) 2024-09-11T00:20:45.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 -- 172.21.15.103:0/4198238030 learned_addr learned my addr 172.21.15.103:0/4198238030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1d0e00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc10d0f0 0x7fd1cc1023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cb400640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc1045b0 0x7fd1cc101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 -- 172.21.15.103:0/4198238030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc10d0f0 msgr2=0x7fd1cc1023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc10d0f0 0x7fd1cc1023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 -- 172.21.15.103:0/4198238030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc1045b0 msgr2=0x7fd1cc101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc1045b0 0x7fd1cc101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 -- 172.21.15.103:0/4198238030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1cc1a76f0 con 0x7fd1cc103c90 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1cbe00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc103c90 0x7fd1cc101940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e9f0 tx=0x7fd1bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.964+0000 7fd1c8c00640 1 -- 172.21.15.103:0/4198238030 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc00cd80 con 0x7fd1cc103c90 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.965+0000 7fd1c8c00640 1 -- 172.21.15.103:0/4198238030 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1bc0108e0 con 0x7fd1cc103c90 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.965+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1cc1a79e0 con 0x7fd1cc103c90 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.965+0000 7fd1c8c00640 1 -- 172.21.15.103:0/4198238030 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1bc018980 con 0x7fd1cc103c90 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.965+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1cc1a7f20 con 0x7fd1cc103c90 2024-09-11T00:20:45.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.966+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1cc104070 con 0x7fd1cc103c90 2024-09-11T00:20:45.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.968+0000 7fd1c8c00640 1 -- 172.21.15.103:0/4198238030 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fd1bc010430 con 0x7fd1cc103c90 2024-09-11T00:20:45.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.969+0000 7fd1c8c00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd1b00b52e0 0x7fd1b00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:45.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.970+0000 7fd1c8c00640 1 -- 172.21.15.103:0/4198238030 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fd1bc0e6160 con 0x7fd1cc103c90 2024-09-11T00:20:45.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.970+0000 7fd1cb400640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd1b00b52e0 0x7fd1b00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:45.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.970+0000 7fd1cb400640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd1b00b52e0 0x7fd1b00b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc103430 tx=0x7fd1b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:45.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:45.974+0000 7fd1c8c00640 1 -- 172.21.15.103:0/4198238030 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd1bc0ac730 con 0x7fd1cc103c90 2024-09-11T00:20:46.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.159+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1cc102c40 con 0x7fd1cc103c90 2024-09-11T00:20:46.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.161+0000 7fd1c8c00640 1 -- 172.21.15.103:0/4198238030 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24244 (secure 0 0 0) 0x7fd1bc0b1160 con 0x7fd1cc103c90 2024-09-11T00:20:46.163 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:46.164 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":85,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:20:41.548880+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:20:46.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.163+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd1b00b52e0 msgr2=0x7fd1b00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:46.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.163+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd1b00b52e0 0x7fd1b00b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc103430 tx=0x7fd1b8005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:46.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.163+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc103c90 msgr2=0x7fd1cc101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:46.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.163+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc103c90 0x7fd1cc101940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd1bc00e9f0 tx=0x7fd1bc00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:20:46.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 shutdown_connections 2024-09-11T00:20:46.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd1b00b52e0 0x7fd1b00b77a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:46.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1cc10d0f0 0x7fd1cc1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:46.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1cc1045b0 0x7fd1cc101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:46.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 --2- 172.21.15.103:0/4198238030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd1cc103c90 0x7fd1cc101940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:46.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 >> 172.21.15.103:0/4198238030 conn(0x7fd1cc0fd710 msgr2=0x7fd1cc0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:46.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 shutdown_connections 2024-09-11T00:20:46.167 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:46.164+0000 7fd1d2e00640 1 -- 172.21.15.103:0/4198238030 wait complete. 2024-09-11T00:20:46.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:46 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4198238030' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:46.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:46 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4198238030' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:46 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4198238030' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:47.013 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:20:47.384 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:47.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:47 smithi103 ceph-mon[30578]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:47.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:47 smithi183 ceph-mon[35934]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:47.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:47 smithi117 ceph-mon[35525]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:48.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2979028267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff10010a800 msgr2=0x7ff10010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:48.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2979028267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff10010a800 0x7ff10010ac40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009900 tx=0x7ff0e802a410 comp rx=0 tx=0).stop 2024-09-11T00:20:48.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2979028267 shutdown_connections 2024-09-11T00:20:48.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2979028267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff10010b180 0x7ff100111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.035 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2979028267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff10010a800 0x7ff10010ac40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2979028267 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff100104df0 0x7ff1001051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2979028267 >> 172.21.15.103:0/2979028267 conn(0x7ff100100840 msgr2=0x7ff100102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:48.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2979028267 shutdown_connections 2024-09-11T00:20:48.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.033+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2979028267 wait complete. 2024-09-11T00:20:48.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.034+0000 7ff0ffe00640 1 Processor -- start 2024-09-11T00:20:48.036 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.034+0000 7ff0ffe00640 1 -- start start 2024-09-11T00:20:48.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff100104df0 0x7ff10019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:48.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff10010a800 0x7ff10019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:48.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0ffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff10010b180 0x7ff1001a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:48.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0ffe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff100114e50 con 0x7ff100104df0 2024-09-11T00:20:48.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0ffe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff100114cd0 con 0x7ff10010a800 2024-09-11T00:20:48.037 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0ffe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff100114fd0 con 0x7ff10010b180 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0f7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff10010a800 0x7ff10019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0f7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff10010a800 0x7ff10019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40480/0 (socket says 172.21.15.103:40480) 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0f7e00640 1 -- 172.21.15.103:0/2154862030 learned_addr learned my addr 172.21.15.103:0/2154862030 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0ff400640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff10010b180 0x7ff1001a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.035+0000 7ff0fea00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff100104df0 0x7ff10019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0ff400640 1 -- 172.21.15.103:0/2154862030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff10010a800 msgr2=0x7ff10019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0ff400640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff10010a800 0x7ff10019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0ff400640 1 -- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff100104df0 msgr2=0x7ff10019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0ff400640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff100104df0 0x7ff10019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.038 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0ff400640 1 -- 172.21.15.103:0/2154862030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1001a12c0 con 0x7ff10010b180 2024-09-11T00:20:48.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0ff400640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff10010b180 0x7ff1001a0af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e940 tx=0x7ff0f800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:48.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0f7400640 1 -- 172.21.15.103:0/2154862030 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004d20 con 0x7ff10010b180 2024-09-11T00:20:48.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1001a15b0 con 0x7ff10010b180 2024-09-11T00:20:48.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.036+0000 7ff0f7400640 1 -- 172.21.15.103:0/2154862030 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f8005020 con 0x7ff10010b180 2024-09-11T00:20:48.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.037+0000 7ff0f7400640 1 -- 172.21.15.103:0/2154862030 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80116d0 con 0x7ff10010b180 2024-09-11T00:20:48.039 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.037+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1001a7c70 con 0x7ff10010b180 2024-09-11T00:20:48.040 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.038+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1001051d0 con 0x7ff10010b180 2024-09-11T00:20:48.041 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.040+0000 7ff0f7400640 1 -- 172.21.15.103:0/2154862030 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7ff0f800f040 con 0x7ff10010b180 2024-09-11T00:20:48.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.041+0000 7ff0f7400640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0e40b5390 0x7ff0e40b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:48.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.041+0000 7ff0f7400640 1 -- 172.21.15.103:0/2154862030 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7ff0f80e5eb0 con 0x7ff10010b180 2024-09-11T00:20:48.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.041+0000 7ff0fea00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0e40b5390 0x7ff0e40b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:48.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.042+0000 7ff0fea00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0e40b5390 0x7ff0e40b7850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00095f0 tx=0x7ff0f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:48.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.043+0000 7ff0f7400640 1 -- 172.21.15.103:0/2154862030 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff0f8016030 con 0x7ff10010b180 2024-09-11T00:20:48.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.235+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff10019c340 con 0x7ff10010b180 2024-09-11T00:20:48.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.236+0000 7ff0f7400640 1 -- 172.21.15.103:0/2154862030 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24244 (secure 0 0 0) 0x7ff0f80ac480 con 0x7ff10010b180 2024-09-11T00:20:48.239 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:48.239 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":85,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:20:41.548880+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:20:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0e40b5390 msgr2=0x7ff0e40b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0e40b5390 0x7ff0e40b7850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0f00095f0 tx=0x7ff0f0005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff10010b180 msgr2=0x7ff1001a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff10010b180 0x7ff1001a0af0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800e940 tx=0x7ff0f800ee00 comp rx=0 tx=0).stop 2024-09-11T00:20:48.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 shutdown_connections 2024-09-11T00:20:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff0e40b5390 0x7ff0e40b7850 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff10010b180 0x7ff1001a0af0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff10010a800 0x7ff10019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 --2- 172.21.15.103:0/2154862030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff100104df0 0x7ff10019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 >> 172.21.15.103:0/2154862030 conn(0x7ff100100840 msgr2=0x7ff100107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 shutdown_connections 2024-09-11T00:20:48.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:48.239+0000 7ff0ffe00640 1 -- 172.21.15.103:0/2154862030 wait complete. 2024-09-11T00:20:48.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:48 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2154862030' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:48.923 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:48 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2154862030' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:48.923 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:48 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2154862030' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:49.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:20:49.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:49 smithi103 ceph-mon[30578]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:49.679 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:49.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:49 smithi117 ceph-mon[35525]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:49.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:49 smithi183 ceph-mon[35934]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 -- 172.21.15.103:0/3850709453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe40776d0 msgr2=0x7f4fe4077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/3850709453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe40776d0 0x7f4fe4077ab0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009900 tx=0x7f4fd402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 -- 172.21.15.103:0/3850709453 shutdown_connections 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/3850709453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe4113990 0x7f4fe4115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/3850709453 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fe4077ff0 0x7f4fe4113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/3850709453 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe40776d0 0x7f4fe4077ab0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 -- 172.21.15.103:0/3850709453 >> 172.21.15.103:0/3850709453 conn(0x7f4fe4100a50 msgr2=0x7f4fe4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 -- 172.21.15.103:0/3850709453 shutdown_connections 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.252+0000 7f4fe9a00640 1 -- 172.21.15.103:0/3850709453 wait complete. 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.253+0000 7f4fe9a00640 1 Processor -- start 2024-09-11T00:20:50.291 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.253+0000 7f4fe9a00640 1 -- start start 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe9a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe40776d0 0x7f4fe419f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe9a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fe4077ff0 0x7f4fe419fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe4113990 0x7f4fe41a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe9a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe41193a0 con 0x7f4fe4113990 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe9a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4119220 con 0x7f4fe40776d0 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe9a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe4119520 con 0x7f4fe4077ff0 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe40776d0 0x7f4fe419f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe40776d0 0x7f4fe419f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40502/0 (socket says 172.21.15.103:40502) 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 -- 172.21.15.103:0/1913631000 learned_addr learned my addr 172.21.15.103:0/1913631000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe3400640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe4113990 0x7f4fe41a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2000640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fe4077ff0 0x7f4fe419fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 -- 172.21.15.103:0/1913631000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fe4077ff0 msgr2=0x7f4fe419fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:50.292 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fe4077ff0 0x7f4fe419fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 -- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe4113990 msgr2=0x7f4fe41a5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe4113990 0x7f4fe41a5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.254+0000 7f4fe2a00640 1 -- 172.21.15.103:0/1913631000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe41a59e0 con 0x7f4fe40776d0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.255+0000 7f4fe2a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe40776d0 0x7f4fe419f930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4005940 tx=0x7f4fd4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.255+0000 7f4fdb400640 1 -- 172.21.15.103:0/1913631000 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd403f030 con 0x7f4fe40776d0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.255+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe41a5c70 con 0x7f4fe40776d0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.255+0000 7f4fdb400640 1 -- 172.21.15.103:0/1913631000 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd4003aa0 con 0x7f4fe40776d0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.255+0000 7f4fdb400640 1 -- 172.21.15.103:0/1913631000 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4035070 con 0x7f4fe40776d0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.255+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe41ac390 con 0x7f4fe40776d0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.256+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe4077ab0 con 0x7f4fe40776d0 2024-09-11T00:20:50.293 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.258+0000 7f4fdb400640 1 -- 172.21.15.103:0/1913631000 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f4fd403bcc0 con 0x7f4fe40776d0 2024-09-11T00:20:50.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.260+0000 7f4fdb400640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4fc80b52e0 0x7f4fc80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:50.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.260+0000 7f4fdb400640 1 -- 172.21.15.103:0/1913631000 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f4fd4106a30 con 0x7f4fe40776d0 2024-09-11T00:20:50.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.260+0000 7f4fe2000640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4fc80b52e0 0x7f4fc80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:50.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.261+0000 7f4fe2000640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4fc80b52e0 0x7f4fc80b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4116010 tx=0x7f4fd000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:50.294 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.261+0000 7f4fdb400640 1 -- 172.21.15.103:0/1913631000 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4fd40cd000 con 0x7f4fe40776d0 2024-09-11T00:20:50.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.484+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fe41ac5c0 con 0x7f4fe40776d0 2024-09-11T00:20:50.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.485+0000 7f4fdb400640 1 -- 172.21.15.103:0/1913631000 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11860 (secure 0 0 0) 0x7f4fd40b2e20 con 0x7f4fe40776d0 2024-09-11T00:20:50.488 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:50.488 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"btime":"2024-09-11T00:20:41:599300+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24599,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24620,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24622,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24628,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24643,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:20:41.549050+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24656},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24656,"qdb_cluster":[24656]},"id":1}]} 2024-09-11T00:20:50.488 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 17 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4fc80b52e0 msgr2=0x7f4fc80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4fc80b52e0 0x7f4fc80b77a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4116010 tx=0x7f4fd000b390 comp rx=0 tx=0).stop 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe40776d0 msgr2=0x7f4fe419f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe40776d0 0x7f4fe419f930 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4005940 tx=0x7f4fd4004270 comp rx=0 tx=0).stop 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 shutdown_connections 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4fc80b52e0 0x7f4fc80b77a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fe4113990 0x7f4fe41a5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fe4077ff0 0x7f4fe419fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 --2- 172.21.15.103:0/1913631000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fe40776d0 0x7f4fe419f930 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 >> 172.21.15.103:0/1913631000 conn(0x7f4fe4100a50 msgr2=0x7f4fe4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 shutdown_connections 2024-09-11T00:20:50.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:50.488+0000 7f4fe9a00640 1 -- 172.21.15.103:0/1913631000 wait complete. 2024-09-11T00:20:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:50 smithi183 ceph-mon[35934]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:50.901 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:50 smithi103 ceph-mon[30578]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:50 smithi117 ceph-mon[35525]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:51.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:20:51.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:51 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1913631000' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:51.891 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:51.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:51 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1913631000' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:51.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:51 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1913631000' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:20:52.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.541+0000 7f18d7600640 1 -- 172.21.15.103:0/824385175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0109150 msgr2=0x7f18d010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:52.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.541+0000 7f18d7600640 1 --2- 172.21.15.103:0/824385175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0109150 0x7f18d010f950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009ca0 tx=0x7f18c802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.541+0000 7f18d7600640 1 -- 172.21.15.103:0/824385175 shutdown_connections 2024-09-11T00:20:52.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.541+0000 7f18d7600640 1 --2- 172.21.15.103:0/824385175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0109150 0x7f18d010f950 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.548 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.541+0000 7f18d7600640 1 --2- 172.21.15.103:0/824385175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18d01087d0 0x7f18d0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.541+0000 7f18d7600640 1 --2- 172.21.15.103:0/824385175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d0102dc0 0x7f18d01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.541+0000 7f18d7600640 1 -- 172.21.15.103:0/824385175 >> 172.21.15.103:0/824385175 conn(0x7f18d00fe830 msgr2=0x7f18d0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.542+0000 7f18d7600640 1 -- 172.21.15.103:0/824385175 shutdown_connections 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.542+0000 7f18d7600640 1 -- 172.21.15.103:0/824385175 wait complete. 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.542+0000 7f18d7600640 1 Processor -- start 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d7600640 1 -- start start 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d7600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0102dc0 0x7f18d00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d7600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01087d0 0x7f18d00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d7600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18d0109150 0x7f18d0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d7600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112dd0 con 0x7f18d0102dc0 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d7600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112c50 con 0x7f18d01087d0 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d7600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0112f50 con 0x7f18d0109150 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0102dc0 0x7f18d00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:52.549 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0102dc0 0x7f18d00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58218/0 (socket says 172.21.15.103:58218) 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d4c00640 1 -- 172.21.15.103:0/3348371489 learned_addr learned my addr 172.21.15.103:0/3348371489 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18cfe00640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01087d0 0x7f18d00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.543+0000 7f18d5600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18d0109150 0x7f18d0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d4c00640 1 -- 172.21.15.103:0/3348371489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18d0109150 msgr2=0x7f18d0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d4c00640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18d0109150 0x7f18d0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d4c00640 1 -- 172.21.15.103:0/3348371489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01087d0 msgr2=0x7f18d00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d4c00640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01087d0 0x7f18d00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d4c00640 1 -- 172.21.15.103:0/3348371489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d01a76a0 con 0x7f18d0102dc0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18cfe00640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01087d0 0x7f18d00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d4c00640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0102dc0 0x7f18d00754a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f18c400ea60 tx=0x7f18c400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18cd600640 1 -- 172.21.15.103:0/3348371489 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c400cde0 con 0x7f18d0102dc0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d01a7990 con 0x7f18d0102dc0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18cd600640 1 -- 172.21.15.103:0/3348371489 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18c40035f0 con 0x7f18d0102dc0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18cd600640 1 -- 172.21.15.103:0/3348371489 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4010690 con 0x7f18d0102dc0 2024-09-11T00:20:52.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.544+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a7ed0 con 0x7f18d0102dc0 2024-09-11T00:20:52.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.545+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d01031a0 con 0x7f18d0102dc0 2024-09-11T00:20:52.551 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.548+0000 7f18cd600640 1 -- 172.21.15.103:0/3348371489 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f18c4004000 con 0x7f18d0102dc0 2024-09-11T00:20:52.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.550+0000 7f18cd600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f18b80b52e0 0x7f18b80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:52.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.550+0000 7f18cd600640 1 -- 172.21.15.103:0/3348371489 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f18c40e6150 con 0x7f18d0102dc0 2024-09-11T00:20:52.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.550+0000 7f18cfe00640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f18b80b52e0 0x7f18b80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:52.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.551+0000 7f18cfe00640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f18b80b52e0 0x7f18b80b77a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18d0076f90 tx=0x7f18bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:52.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.554+0000 7f18cd600640 1 -- 172.21.15.103:0/3348371489 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f18c40acd70 con 0x7f18d0102dc0 2024-09-11T00:20:52.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.739+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18d00767a0 con 0x7f18d0102dc0 2024-09-11T00:20:52.741 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:52 smithi103 ceph-mon[30578]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:52.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.740+0000 7f18cd600640 1 -- 172.21.15.103:0/3348371489 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24244 (secure 0 0 0) 0x7f18c40ac670 con 0x7f18d0102dc0 2024-09-11T00:20:52.743 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:20:52.743 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":85,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:20:41.548880+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:20:52.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f18b80b52e0 msgr2=0x7f18b80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:52.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f18b80b52e0 0x7f18b80b77a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f18d0076f90 tx=0x7f18bc005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:52.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0102dc0 msgr2=0x7f18d00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:52.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0102dc0 0x7f18d00754a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f18c400ea60 tx=0x7f18c400ef20 comp rx=0 tx=0).stop 2024-09-11T00:20:52.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 shutdown_connections 2024-09-11T00:20:52.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f18b80b52e0 0x7f18b80b77a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f18d0109150 0x7f18d0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18d01087d0 0x7f18d00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 --2- 172.21.15.103:0/3348371489 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18d0102dc0 0x7f18d00754a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:52.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 >> 172.21.15.103:0/3348371489 conn(0x7f18d00fe830 msgr2=0x7f18d0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:52.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.743+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 shutdown_connections 2024-09-11T00:20:52.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:52.744+0000 7f18d7600640 1 -- 172.21.15.103:0/3348371489 wait complete. 2024-09-11T00:20:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:52 smithi183 ceph-mon[35934]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:52.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:52 smithi117 ceph-mon[35525]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:53.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:53 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3348371489' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:53.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T00:20:28.229329+0000', 'modified': '2024-09-11T00:20:41.549050+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24656}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24656': {'gid': 24656, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/3769588373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 3769588373}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 3769588373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24656, 'qdb_cluster': [24656]} 2024-09-11T00:20:53.748 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T00:20:53.748 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:20:53.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:53 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3348371489' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:53 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3348371489' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:20:54.119 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:54.616 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:54 smithi103 ceph-mon[30578]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 -- 172.21.15.103:0/1322009707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 msgr2=0x7f548c10ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 --2- 172.21.15.103:0/1322009707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c10ee10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5474009900 tx=0x7f547402a410 comp rx=0 tx=0).stop 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 -- 172.21.15.103:0/1322009707 shutdown_connections 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 --2- 172.21.15.103:0/1322009707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f548c10f3e0 0x7f548c1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 --2- 172.21.15.103:0/1322009707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c10ee10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 --2- 172.21.15.103:0/1322009707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c075540 0x7f548c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 -- 172.21.15.103:0/1322009707 >> 172.21.15.103:0/1322009707 conn(0x7f548c0fe830 msgr2=0x7f548c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 -- 172.21.15.103:0/1322009707 shutdown_connections 2024-09-11T00:20:54.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.725+0000 7f5492800640 1 -- 172.21.15.103:0/1322009707 wait complete. 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 Processor -- start 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 -- start start 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c075540 0x7f548c19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f548c10f3e0 0x7f548c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f548c114fd0 con 0x7f548c075f40 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f548c114e50 con 0x7f548c075540 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.727+0000 7f5492800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f548c115150 con 0x7f548c10f3e0 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58242/0 (socket says 172.21.15.103:58242) 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 -- 172.21.15.103:0/148314972 learned_addr learned my addr 172.21.15.103:0/148314972 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f548b400640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c075540 0x7f548c19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f548be00640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f548c10f3e0 0x7f548c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 -- 172.21.15.103:0/148314972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f548c10f3e0 msgr2=0x7f548c1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f548c10f3e0 0x7f548c1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 -- 172.21.15.103:0/148314972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c075540 msgr2=0x7f548c19b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c075540 0x7f548c19b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 -- 172.21.15.103:0/148314972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c1a1590 con 0x7f548c075f40 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.728+0000 7f5483e00640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c19ba20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5474005ab0 tx=0x7f5474003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.729+0000 7f5488c00640 1 -- 172.21.15.103:0/148314972 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547403f070 con 0x7f548c075f40 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.729+0000 7f5488c00640 1 -- 172.21.15.103:0/148314972 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5474031b60 con 0x7f548c075f40 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.729+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c1a1820 con 0x7f548c075f40 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.729+0000 7f5488c00640 1 -- 172.21.15.103:0/148314972 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474035030 con 0x7f548c075f40 2024-09-11T00:20:54.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.729+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c1a7f40 con 0x7f548c075f40 2024-09-11T00:20:54.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 Processor -- start 2024-09-11T00:20:54.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 -- start start 2024-09-11T00:20:54.737 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f5488c00640 1 -- 172.21.15.103:0/148314972 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f547403b460 con 0x7f548c075f40 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa13a240 0x557aaa13a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa1465b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa146af0 0x557aaa148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557aaa0b1d50 con 0x557aaa146af0 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557aaa0a1180 con 0x557aaa13a240 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f549528d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557aaa0a4f90 con 0x557aaa13ab60 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f548b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa1465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f548b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa1465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:58624/0 (socket says 172.21.15.103:58624) 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f548b400640 1 -- 172.21.15.103:0/646236998 learned_addr learned my addr 172.21.15.103:0/646236998 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.732+0000 7f5483e00640 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa146af0 0x557aaa148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.738 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548be00640 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa13a240 0x557aaa13a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548b400640 1 -- 172.21.15.103:0/646236998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa13a240 msgr2=0x557aaa13a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548b400640 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa13a240 0x557aaa13a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548b400640 1 -- 172.21.15.103:0/646236998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa146af0 msgr2=0x557aaa148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548b400640 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa146af0 0x557aaa148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548b400640 1 -- 172.21.15.103:0/646236998 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aaa06ba70 con 0x557aaa13ab60 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548be00640 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa13a240 0x557aaa13a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f548b400640 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa1465b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f547c009d20 tx=0x7f547c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db150bb91918c156 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.733+0000 7f5480c00640 1 -- 172.21.15.103:0/646236998 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c004230 con 0x557aaa13ab60 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f5480c00640 1 -- 172.21.15.103:0/646236998 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f547c0043d0 con 0x557aaa13ab60 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f5480c00640 1 -- 172.21.15.103:0/646236998 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c0197c0 con 0x557aaa13ab60 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 -- 172.21.15.103:0/646236998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 msgr2=0x557aaa1465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.739 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa1465b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f547c009d20 tx=0x7f547c011810 comp rx=0 tx=0).stop 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 -- 172.21.15.103:0/646236998 shutdown_connections 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa146af0 0x557aaa148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa1465b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 --2- 172.21.15.103:0/646236998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa13a240 0x557aaa13a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 -- 172.21.15.103:0/646236998 >> 172.21.15.103:0/646236998 conn(0x557aaa138020 msgr2=0x557aaa139210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f5488c00640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f54700b5360 0x7f54700b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.734+0000 7f549528d740 1 -- 172.21.15.103:0/646236998 shutdown_connections 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.735+0000 7f549528d740 1 -- 172.21.15.103:0/646236998 wait complete. 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.735+0000 7f5488c00640 1 -- 172.21.15.103:0/148314972 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f5474106770 con 0x7f548c075f40 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.735+0000 7f548be00640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f54700b5360 0x7f54700b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.735+0000 7f549528d740 1 -- start start 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.735+0000 7f548be00640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f54700b5360 0x7f54700b7820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f548c06c0b0 tx=0x7f5484009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:54.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.735+0000 7f549528d740 1 Processor -- start 2024-09-11T00:20:54.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f549528d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13a240 0x557aaa146540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f549528d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa143990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f549528d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa144120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f549528d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557aa9f025f0 con 0x557aaa13a240 2024-09-11T00:20:54.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f549528d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557aaa03c460 con 0x557aaa146af0 2024-09-11T00:20:54.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f549528d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557aaa149840 con 0x557aaa13ab60 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f549528d740 10 client.0 _ll_register_callbacks cb 0x557aaa04c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f548b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13a240 0x557aaa146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f5483e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f5483e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:58630/0 (socket says 172.21.15.103:58630) 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f5483e00640 1 -- 172.21.15.103:0/2717150628 learned_addr learned my addr 172.21.15.103:0/2717150628 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f548b400640 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13a240 0x557aaa146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58268/0 (socket says 172.21.15.103:58268) 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.736+0000 7f548be00640 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5483e00640 1 -- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 msgr2=0x557aaa144120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5483e00640 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa144120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5483e00640 1 -- 172.21.15.103:0/2717150628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13a240 msgr2=0x557aaa146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5483e00640 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13a240 0x557aaa146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5483e00640 1 -- 172.21.15.103:0/2717150628 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa9dff540 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5483e00640 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa143990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54740c9310 tx=0x7f54740c9390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64c54a3596028393 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5479600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5479600640 1 -- 172.21.15.103:0/2717150628 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474049070 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557aaa03ea70 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5479600640 1 -- 172.21.15.103:0/2717150628 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f54740ccfa0 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.737+0000 7f5479600640 1 -- 172.21.15.103:0/2717150628 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f547410c020 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 10 client.24775 fetch_fsmap learned FSMap version 17 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x557aaa0596f0 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f5479600640 1 -- 172.21.15.103:0/2717150628 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557aaa03ea70 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f5479600640 1 -- 172.21.15.103:0/2717150628 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f5474109ce0 con 0x557aaa13ab60 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 10 client.24775 fetch_fsmap finished waiting for FSMap version 17 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 10 client.24775 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 20 client.24775 populate_metadata read hostname 'smithi103' 2024-09-11T00:20:54.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa157800 0x557aaa158c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 4 client.24775 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.738+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aaa0b3ca0 con 0x557aaa157800 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.739+0000 7f548b400640 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa157800 0x557aaa158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.739+0000 7f548b400640 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa157800 0x557aaa158c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e0bbd3811637d9 server_cookie=35a48b9f3503b4ee in_seq=0 out_seq=0 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.739+0000 7f5479600640 10 client.24775 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f5479600640 1 -- 172.21.15.103:0/2717150628 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557aaa0b3ca0 con 0x557aaa157800 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f5479600640 10 client.24775 handle_command_reply: tid=0 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 client.24775 shutdown 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 msgr2=0x557aaa143990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa143990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f54740c9310 tx=0x7f54740c9390 comp rx=0 tx=0).stop 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 shutdown_connections 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa144120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa157800 0x557aaa158c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13ab60 0x557aaa143990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 --2- 172.21.15.103:0/2717150628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13a240 0x557aaa146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 >> 172.21.15.103:0/2717150628 conn(0x557aaa138020 msgr2=0x557aaa13dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.740+0000 7f5479600640 0 client.24775 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:20:54.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.741+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 shutdown_connections 2024-09-11T00:20:54.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.741+0000 7f549528d740 1 -- 172.21.15.103:0/2717150628 wait complete. 2024-09-11T00:20:54.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.741+0000 7f549528d740 20 client.24775 trim_cache size 0 max 16384 2024-09-11T00:20:54.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.759+0000 7f549528d740 1 Processor -- start 2024-09-11T00:20:54.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f549528d740 1 -- start start 2024-09-11T00:20:54.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f549528d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f549528d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f549528d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa157130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f549528d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557aaa156fb0 con 0x557aaa13ab60 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f549528d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557aaa157680 con 0x557aaa146af0 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f549528d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557aaa157d50 con 0x557aaa13a240 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f5483e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f548b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f5483e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58276/0 (socket says 172.21.15.103:58276) 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f548b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:58644/0 (socket says 172.21.15.103:58644) 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f5483e00640 1 -- 172.21.15.103:0/1829357748 learned_addr learned my addr 172.21.15.103:0/1829357748 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f548be00640 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.760+0000 7f548b400640 1 -- 172.21.15.103:0/1829357748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 msgr2=0x557aaa157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f548b400640 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f548b400640 1 -- 172.21.15.103:0/1829357748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 msgr2=0x557aaa1585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f548b400640 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f548b400640 1 -- 172.21.15.103:0/1829357748 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aaa13a990 con 0x557aaa13a240 2024-09-11T00:20:54.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f548be00640 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:20:54.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f5483e00640 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:20:54.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f548b400640 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa157ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f547c0096e0 tx=0x7f547c003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcf09f11bf1f016 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:54.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f547be00640 1 -- 172.21.15.103:0/1829357748 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c031070 con 0x557aaa13a240 2024-09-11T00:20:54.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f547be00640 1 -- 172.21.15.103:0/1829357748 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f547c0035f0 con 0x557aaa13a240 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.761+0000 7f547be00640 1 -- 172.21.15.103:0/1829357748 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c019710 con 0x557aaa13a240 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 -- 172.21.15.103:0/1829357748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 msgr2=0x557aaa157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa157ed0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f547c0096e0 tx=0x7f547c003ef0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 -- 172.21.15.103:0/1829357748 shutdown_connections 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 --2- 172.21.15.103:0/1829357748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa157ed0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 -- 172.21.15.103:0/1829357748 >> 172.21.15.103:0/1829357748 conn(0x557aaa138020 msgr2=0x557aaa1564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 -- 172.21.15.103:0/1829357748 shutdown_connections 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 -- 172.21.15.103:0/1829357748 wait complete. 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 -- start start 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.762+0000 7f549528d740 1 Processor -- start 2024-09-11T00:20:54.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f549528d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f549528d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f549528d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa160600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f549528d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557aaa158af0 con 0x557aaa13ab60 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f549528d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x557aaa158420 con 0x557aaa146af0 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f549528d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557aaa1591c0 con 0x557aaa13a240 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f549528d740 10 client.0 _ll_register_callbacks cb 0x7f5474009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f5483e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f5483e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58286/0 (socket says 172.21.15.103:58286) 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f5483e00640 1 -- 172.21.15.103:0/640558187 learned_addr learned my addr 172.21.15.103:0/640558187 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f548b400640 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f548be00640 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f5483e00640 1 -- 172.21.15.103:0/640558187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 msgr2=0x557aaa161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f5483e00640 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f5483e00640 1 -- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 msgr2=0x557aaa160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:20:54.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.763+0000 7f5483e00640 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f5483e00640 1 -- 172.21.15.103:0/640558187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aaa160b90 con 0x557aaa13ab60 2024-09-11T00:20:54.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f5483e00640 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1600c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54740c9310 tx=0x7f54740c91e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35312e078ac5de87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:54.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f5479600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f5479600640 1 -- 172.21.15.103:0/640558187 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474049070 con 0x557aaa13ab60 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f5479600640 1 -- 172.21.15.103:0/640558187 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f54740b2b60 con 0x557aaa13ab60 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f547c0035f0 con 0x557aaa13ab60 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f5479600640 1 -- 172.21.15.103:0/640558187 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f54740cc850 con 0x557aaa13ab60 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f549528d740 10 client.14748 fetch_fsmap learned FSMap version 17 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.764+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x557aaa15fde0 con 0x557aaa13ab60 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f5479600640 1 -- 172.21.15.103:0/640558187 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f547c0035f0 con 0x557aaa13ab60 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f5479600640 1 -- 172.21.15.103:0/640558187 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f54740cbfa0 con 0x557aaa13ab60 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f549528d740 10 client.14748 fetch_fsmap finished waiting for FSMap version 17 2024-09-11T00:20:54.770 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f549528d740 10 client.14748 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f549528d740 20 client.14748 populate_metadata read hostname 'smithi103' 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f549528d740 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa1c8950 0x557aaa1c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f549528d740 4 client.14748 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "status"}) -- 0x557aaa0af750 con 0x557aaa1c8950 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.765+0000 7f548b400640 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa1c8950 0x557aaa1c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f548b400640 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa1c8950 0x557aaa1c7660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7596e6efb5cd523d server_cookie=6e8e1371a9066cf4 in_seq=0 out_seq=0 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f5479600640 10 client.14748 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f5479600640 1 -- 172.21.15.103:0/640558187 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x557aaa0af750 con 0x557aaa1c8950 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f5479600640 10 client.14748 handle_command_reply: tid=0 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 client.14748 shutdown 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 msgr2=0x557aaa1600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1600c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f54740c9310 tx=0x7f54740c91e0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 shutdown_connections 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x557aaa146af0 0x557aaa160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.771 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557aaa13ab60 0x557aaa1600c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557aaa1c8950 0x557aaa1c7660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 --2- 172.21.15.103:0/640558187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557aaa13a240 0x557aaa161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 >> 172.21.15.103:0/640558187 conn(0x557aaa138020 msgr2=0x557aaa139180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f5479600640 0 client.14748 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 shutdown_connections 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.766+0000 7f549528d740 1 -- 172.21.15.103:0/640558187 wait complete. 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f549528d740 20 client.14748 trim_cache size 0 max 16384 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-11T00:20:54.772 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24656, 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 25.712617793, 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 17, 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 85, 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 84, 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 40.688698844000001 2024-09-11T00:20:54.773 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f54700b5360 msgr2=0x7f54700b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f54700b5360 0x7f54700b7820 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f548c06c0b0 tx=0x7f5484009500 comp rx=0 tx=0).stop 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 msgr2=0x7f548c19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c19ba20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5474005ab0 tx=0x7f5474003c10 comp rx=0 tx=0).stop 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 shutdown_connections 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f54700b5360 0x7f54700b7820 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f548c10f3e0 0x7f548c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f548c075f40 0x7f548c19ba20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 --2- 172.21.15.103:0/148314972 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f548c075540 0x7f548c19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.767+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 >> 172.21.15.103:0/148314972 conn(0x7f548c0fe830 msgr2=0x7f548c1115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.768+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 shutdown_connections 2024-09-11T00:20:54.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:54.768+0000 7f5492800640 1 -- 172.21.15.103:0/148314972 wait complete. 2024-09-11T00:20:54.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:54 smithi183 ceph-mon[35934]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:54 smithi117 ceph-mon[35525]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:55.725 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T00:20:55.725 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:20:55.725 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T00:20:56.096 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:20:56.608 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:56 smithi103 ceph-mon[30578]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:56.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 -- 172.21.15.103:0/2555641601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 msgr2=0x7f733c0758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 --2- 172.21.15.103:0/2555641601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c0758a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f732c009900 tx=0x7f732c02a410 comp rx=0 tx=0).stop 2024-09-11T00:20:56.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 -- 172.21.15.103:0/2555641601 shutdown_connections 2024-09-11T00:20:56.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 --2- 172.21.15.103:0/2555641601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f733c10f3b0 0x7f733c1117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 --2- 172.21.15.103:0/2555641601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c075de0 0x7f733c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 --2- 172.21.15.103:0/2555641601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c0758a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 -- 172.21.15.103:0/2555641601 >> 172.21.15.103:0/2555641601 conn(0x7f733c0fe810 msgr2=0x7f733c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:56.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 -- 172.21.15.103:0/2555641601 shutdown_connections 2024-09-11T00:20:56.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.716+0000 7f7342600640 1 -- 172.21.15.103:0/2555641601 wait complete. 2024-09-11T00:20:56.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.717+0000 7f7342600640 1 Processor -- start 2024-09-11T00:20:56.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.717+0000 7f7342600640 1 -- start start 2024-09-11T00:20:56.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f7342600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c19b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f7342600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c075de0 0x7f733c19b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f7342600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f733c10f3b0 0x7f733c1a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f7342600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f733c114fe0 con 0x7f733c0754c0 2024-09-11T00:20:56.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f7342600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f733c114e60 con 0x7f733c075de0 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f7342600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f733c115160 con 0x7f733c10f3b0 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c19b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c19b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58298/0 (socket says 172.21.15.103:58298) 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 -- 172.21.15.103:0/3779917061 learned_addr learned my addr 172.21.15.103:0/3779917061 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733aa00640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c075de0 0x7f733c19b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733be00640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f733c10f3b0 0x7f733c1a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 -- 172.21.15.103:0/3779917061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f733c10f3b0 msgr2=0x7f733c1a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f733c10f3b0 0x7f733c1a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 -- 172.21.15.103:0/3779917061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c075de0 msgr2=0x7f733c19b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c075de0 0x7f733c19b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.718+0000 7f733b400640 1 -- 172.21.15.103:0/3779917061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c1a1540 con 0x7f733c0754c0 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.719+0000 7f733b400640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c19b490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f732c005ab0 tx=0x7f732c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.719+0000 7f7333e00640 1 -- 172.21.15.103:0/3779917061 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c03f070 con 0x7f733c0754c0 2024-09-11T00:20:56.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.719+0000 7f7333e00640 1 -- 172.21.15.103:0/3779917061 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f732c031b60 con 0x7f733c0754c0 2024-09-11T00:20:56.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.719+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c1a17d0 con 0x7f733c0754c0 2024-09-11T00:20:56.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.719+0000 7f7333e00640 1 -- 172.21.15.103:0/3779917061 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c035030 con 0x7f733c0754c0 2024-09-11T00:20:56.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.719+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c1a7ef0 con 0x7f733c0754c0 2024-09-11T00:20:56.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.720+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c0758a0 con 0x7f733c0754c0 2024-09-11T00:20:56.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.722+0000 7f7333e00640 1 -- 172.21.15.103:0/3779917061 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f732c03b460 con 0x7f733c0754c0 2024-09-11T00:20:56.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.724+0000 7f7333e00640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f73200b52e0 0x7f73200b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:20:56.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.724+0000 7f7333e00640 1 -- 172.21.15.103:0/3779917061 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f732c1067c0 con 0x7f733c0754c0 2024-09-11T00:20:56.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.724+0000 7f733aa00640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f73200b52e0 0x7f73200b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:20:56.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.724+0000 7f733aa00640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f73200b52e0 0x7f73200b77a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f733c101040 tx=0x7f7328005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:20:56.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.728+0000 7f7333e00640 1 -- 172.21.15.103:0/3779917061 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f732c0ccd90 con 0x7f733c0754c0 2024-09-11T00:20:56.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:56 smithi183 ceph-mon[35934]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:56 smithi117 ceph-mon[35525]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.948+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f733c1a8150 con 0x7f733c0754c0 2024-09-11T00:20:56.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.948+0000 7f7333e00640 1 -- 172.21.15.103:0/3779917061 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f732c0d17c0 con 0x7f733c0754c0 2024-09-11T00:20:56.950 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f73200b52e0 msgr2=0x7f73200b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f73200b52e0 0x7f73200b77a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f733c101040 tx=0x7f7328005f90 comp rx=0 tx=0).stop 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 msgr2=0x7f733c19b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c19b490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f732c005ab0 tx=0x7f732c003c10 comp rx=0 tx=0).stop 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 shutdown_connections 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f73200b52e0 0x7f73200b77a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f733c10f3b0 0x7f733c1a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f733c075de0 0x7f733c19b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 --2- 172.21.15.103:0/3779917061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f733c0754c0 0x7f733c19b490 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:20:56.953 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 >> 172.21.15.103:0/3779917061 conn(0x7f733c0fe810 msgr2=0x7f733c111570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:20:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 shutdown_connections 2024-09-11T00:20:56.954 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:56.951+0000 7f7342600640 1 -- 172.21.15.103:0/3779917061 wait complete. 2024-09-11T00:20:57.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:57 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3779917061' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:20:57.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:57 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3779917061' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T00:20:57.785 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-11T00:20:57.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:20:57.786 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:20:57.822 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42142sec preferred_lft 42142sec 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-11T00:20:57.823 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:20:57.825 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T00:20:57.825 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:20:57.825 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:20:57.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-09-11T00:20:57.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-09-11T00:20:57.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-09-11T00:20:57.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T00:20:57.825 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:20:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:57 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3779917061' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:20:57.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:20:58.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:20:58.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:20:58.251 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T00:20:58.308 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-09-11T00:20:58.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:20:58.312 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-09-11T00:20:58.393 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-09-11T00:20:58.393 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T00:20:58.393 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T00:20:58.393 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T00:20:58.393 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T00:20:58.394 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:20:58.394 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:20:58.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T00:20:58.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T00:20:58.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T00:20:58.394 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:20:58.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:20:58.718 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:20:58 smithi103 ceph-mon[30578]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:20:58 smithi183 ceph-mon[35934]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:20:58 smithi117 ceph-mon[35525]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:20:59.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:20:59.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:20:59.023 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-11T00:20:59.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:20:59.060 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-09-11T00:20:59.136 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:20:59.136 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:20:59.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:20:59.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-11T00:20:59.136 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:20:59.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:20:59.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:20:59.315 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-11T00:20:59.315 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:20:59.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:20:59.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:20:59.566 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:20:59.566 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:20:59.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-09-11T00:20:59.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T00:20:59.566 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:20:59.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:20:59.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:20:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:20:59.738 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-11T00:20:59.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:20:59.738 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:20:59.773 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-11T00:20:59.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:20:59.774 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:20:59.848 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-09-11T00:20:59.946 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T00:20:59.982 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-09-11T00:20:59.983 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/b81568fa915b63da1e5a699b781063042766135e4921471a2abdf3d13eec7aa0/merged 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/49c5f4f148f602158ee89e1b52d94ed3fac21e17e6a9069e7fffd0a1b4200a2b/merged 2024-09-11T00:20:59.984 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/7670bd937cd1f8f9cc73cb7aa7888f82346fbb158db3fd80cf8c1e84768d720e/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/33abeed5ac1d3c09b076db6f92974d875445b13a6acf4418019a682c03ad1e43/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a484abccfe00855182302047e6ef2bd0061e3d5e5d4f9c4ffe77d49fdd3130a7/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/3121e35890d2da852bdc14db9486edc4d1e6b046fc26ad3fb8f21b25833574a8/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/c8068a462ceb2fdcd6ead3fd3eba2de51109dbce1efc0f394058316717f70343/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/2c8674d67f1fcbc967daa546a177c254311d47a0dd3682839355cfe02ac35103/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/5896f5512bdf8aabc0455caa95655c1c2104b89404be6366986e33dd8dd18131/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/0002b726be41bb21893045897ef77198c018d82a8ddbb15afcd6fed7834be8ae/merged 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:20:59.985 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:20:59.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:20:59.986 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-09-11T00:21:00.061 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-11T00:21:00.062 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-11T00:21:00.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-09-11T00:21:00.162 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T00:21:00.198 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-09-11T00:21:00.198 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-09-11T00:21:00.198 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-09-11T00:21:00.198 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-09-11T00:21:00.199 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/b81568fa915b63da1e5a699b781063042766135e4921471a2abdf3d13eec7aa0/merged 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/49c5f4f148f602158ee89e1b52d94ed3fac21e17e6a9069e7fffd0a1b4200a2b/merged 2024-09-11T00:21:00.200 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/7670bd937cd1f8f9cc73cb7aa7888f82346fbb158db3fd80cf8c1e84768d720e/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/33abeed5ac1d3c09b076db6f92974d875445b13a6acf4418019a682c03ad1e43/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a484abccfe00855182302047e6ef2bd0061e3d5e5d4f9c4ffe77d49fdd3130a7/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/3121e35890d2da852bdc14db9486edc4d1e6b046fc26ad3fb8f21b25833574a8/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/c8068a462ceb2fdcd6ead3fd3eba2de51109dbce1efc0f394058316717f70343/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/2c8674d67f1fcbc967daa546a177c254311d47a0dd3682839355cfe02ac35103/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/5896f5512bdf8aabc0455caa95655c1c2104b89404be6366986e33dd8dd18131/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/0002b726be41bb21893045897ef77198c018d82a8ddbb15afcd6fed7834be8ae/merged 2024-09-11T00:21:00.201 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-09-11T00:21:00.202 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:21:00.202 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:21:00.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:21:00.203 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-09-11T00:21:00.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:2024-09-11T00:21:00.334+0000 7f4c8b31f480 -1 init, newargv = 0x55cc835f3340 newargc=15 2024-09-11T00:21:00.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:ceph-fuse[76454]: starting ceph client 2024-09-11T00:21:00.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:ceph-fuse[76454]: starting fuse 2024-09-11T00:21:00.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:00 smithi103 ceph-mon[30578]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:00 smithi183 ceph-mon[35934]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:00 smithi117 ceph-mon[35525]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:01.380 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-09-11T00:21:01.446 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-09-11T00:21:01.485 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-09-11T00:21:01.486 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-09-11T00:21:01.487 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/b81568fa915b63da1e5a699b781063042766135e4921471a2abdf3d13eec7aa0/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/49c5f4f148f602158ee89e1b52d94ed3fac21e17e6a9069e7fffd0a1b4200a2b/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/0 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/7670bd937cd1f8f9cc73cb7aa7888f82346fbb158db3fd80cf8c1e84768d720e/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/33abeed5ac1d3c09b076db6f92974d875445b13a6acf4418019a682c03ad1e43/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/a484abccfe00855182302047e6ef2bd0061e3d5e5d4f9c4ffe77d49fdd3130a7/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/3121e35890d2da852bdc14db9486edc4d1e6b046fc26ad3fb8f21b25833574a8/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/c8068a462ceb2fdcd6ead3fd3eba2de51109dbce1efc0f394058316717f70343/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/2c8674d67f1fcbc967daa546a177c254311d47a0dd3682839355cfe02ac35103/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/5896f5512bdf8aabc0455caa95655c1c2104b89404be6366986e33dd8dd18131/merged 2024-09-11T00:21:01.488 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/containers/storage/overlay/0002b726be41bb21893045897ef77198c018d82a8ddbb15afcd6fed7834be8ae/merged 2024-09-11T00:21:01.489 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-09-11T00:21:01.489 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:21:01.489 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:21:01.489 INFO:teuthology.orchestra.run.smithi103.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-11T00:21:01.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:21:01.490 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-09-11T00:21:01.565 INFO:teuthology.orchestra.run.smithi103.stdout:120 2024-09-11T00:21:01.565 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-11T00:21:01.566 DEBUG:teuthology.orchestra.run.smithi103:> sudo stdin-killer -- python3 -c ' 2024-09-11T00:21:01.566 DEBUG:teuthology.orchestra.run.smithi103:> import glob 2024-09-11T00:21:01.566 DEBUG:teuthology.orchestra.run.smithi103:> import re 2024-09-11T00:21:01.566 DEBUG:teuthology.orchestra.run.smithi103:> import os 2024-09-11T00:21:01.566 DEBUG:teuthology.orchestra.run.smithi103:> import subprocess 2024-09-11T00:21:01.566 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> def _find_admin_socket(client_name): 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> files = glob.glob(asok_path) 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> # Given a non-glob path, it better be there 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> if "*" not in asok_path: 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> assert(len(files) == 1) 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> return files[0] 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> for f in files: 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-11T00:21:01.567 DEBUG:teuthology.orchestra.run.smithi103:> contents = proc_f.read() 2024-09-11T00:21:01.568 DEBUG:teuthology.orchestra.run.smithi103:> if mountpoint in contents: 2024-09-11T00:21:01.568 DEBUG:teuthology.orchestra.run.smithi103:> return f 2024-09-11T00:21:01.568 DEBUG:teuthology.orchestra.run.smithi103:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-11T00:21:01.568 DEBUG:teuthology.orchestra.run.smithi103:> 2024-09-11T00:21:01.568 DEBUG:teuthology.orchestra.run.smithi103:> print(_find_admin_socket("client.admin")) 2024-09-11T00:21:01.568 DEBUG:teuthology.orchestra.run.smithi103:> ' 2024-09-11T00:21:01.728 INFO:teuthology.orchestra.run.smithi103.stdout:/var/run/ceph/ceph-client.admin.76454.asok 2024-09-11T00:21:01.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:21:01.742 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76454.asok 2024-09-11T00:21:01.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:21:01.743 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76454.asok status 2024-09-11T00:21:01.927 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:21:01.927 INFO:teuthology.orchestra.run.smithi103.stdout: "metadata": { 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "entity_id": "admin", 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": "smithi103", 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "pid": "76454", 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "root": "/" 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_count": 0, 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_pinned_count": 0, 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 14760, 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "inst": { 2024-09-11T00:21:01.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": { 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client", 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "num": 14760 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3949528929 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3949528929 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:21:01.929 INFO:teuthology.orchestra.run.smithi103.stdout: "inst_str": "client.14760 192.168.0.1:0/3949528929", 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout: "addr_str": "192.168.0.1:0/3949528929", 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout: "inode_count": 1, 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_epoch": 17, 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch": 85, 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch_barrier": 0, 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout: "blocklisted": false, 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs" 2024-09-11T00:21:01.930 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:21:01.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:21:01.940 DEBUG:teuthology.orchestra.run.smithi103:> stat --file-system '--printf=%T 2024-09-11T00:21:01.940 DEBUG:teuthology.orchestra.run.smithi103:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:21:01.977 INFO:teuthology.orchestra.run.smithi103.stdout:fuseblk 2024-09-11T00:21:01.977 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:21:01.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:21:01.977 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:21:02.075 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-11T00:21:02.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:21:02.184 INFO:teuthology.orchestra.run.smithi103.stdout:# file: . 2024-09-11T00:21:02.184 INFO:teuthology.orchestra.run.smithi103.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-11T00:21:02.184 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:21:02.192 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:21:02.192 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:21:02.229 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T00:21:02.647 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:02.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:02 smithi103 ceph-mon[30578]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:02 smithi183 ceph-mon[35934]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:02 smithi117 ceph-mon[35525]: pgmap v333: 129 pgs: 129 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 -- 172.21.15.103:0/1780356222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 msgr2=0x7fdffc10ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 --2- 172.21.15.103:0/1780356222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc10ee90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009900 tx=0x7fdfe802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 -- 172.21.15.103:0/1780356222 shutdown_connections 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 --2- 172.21.15.103:0/1780356222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc10f3d0 0x7fdffc1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 --2- 172.21.15.103:0/1780356222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc10ee90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 --2- 172.21.15.103:0/1780356222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdffc0754c0 0x7fdffc0758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 -- 172.21.15.103:0/1780356222 >> 172.21.15.103:0/1780356222 conn(0x7fdffc0fe830 msgr2=0x7fdffc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:03.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 -- 172.21.15.103:0/1780356222 shutdown_connections 2024-09-11T00:21:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.383+0000 7fe002800640 1 -- 172.21.15.103:0/1780356222 wait complete. 2024-09-11T00:21:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.384+0000 7fe002800640 1 Processor -- start 2024-09-11T00:21:03.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.384+0000 7fe002800640 1 -- start start 2024-09-11T00:21:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fe002800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdffc0754c0 0x7fdffc19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fe002800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fe002800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc10f3d0 0x7fdffc1a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fe002800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc115000 con 0x7fdffc075de0 2024-09-11T00:21:03.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fe002800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc114e80 con 0x7fdffc10f3d0 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fe002800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdffc115180 con 0x7fdffc0754c0 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fdffaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fdffb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdffc0754c0 0x7fdffc19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fdffaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc19ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44076/0 (socket says 172.21.15.103:44076) 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fdffaa00640 1 -- 172.21.15.103:0/3177043438 learned_addr learned my addr 172.21.15.103:0/3177043438 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.385+0000 7fdffbe00640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc10f3d0 0x7fdffc1a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdffb400640 1 -- 172.21.15.103:0/3177043438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc10f3d0 msgr2=0x7fdffc1a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:03.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdffb400640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc10f3d0 0x7fdffc1a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdffb400640 1 -- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 msgr2=0x7fdffc19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdffb400640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdffb400640 1 -- 172.21.15.103:0/3177043438 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdffc1a1580 con 0x7fdffc0754c0 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdffaa00640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdffb400640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdffc0754c0 0x7fdffc19b4d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00e9f0 tx=0x7fdfec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdff3e00640 1 -- 172.21.15.103:0/3177043438 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec00ccc0 con 0x7fdffc0754c0 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdffc1a1870 con 0x7fdffc0754c0 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdff3e00640 1 -- 172.21.15.103:0/3177043438 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfec0108e0 con 0x7fdffc0754c0 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fdff3e00640 1 -- 172.21.15.103:0/3177043438 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec018880 con 0x7fdffc0754c0 2024-09-11T00:21:03.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.386+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdffc1a8100 con 0x7fdffc0754c0 2024-09-11T00:21:03.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.387+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdffc0769d0 con 0x7fdffc0754c0 2024-09-11T00:21:03.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.390+0000 7fdff3e00640 1 -- 172.21.15.103:0/3177043438 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fdfec010430 con 0x7fdffc0754c0 2024-09-11T00:21:03.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.391+0000 7fdff3e00640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdfe00b52e0 0x7fdfe00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:03.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.391+0000 7fdff3e00640 1 -- 172.21.15.103:0/3177043438 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fdfec0e6640 con 0x7fdffc0754c0 2024-09-11T00:21:03.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.391+0000 7fdffaa00640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdfe00b52e0 0x7fdfe00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:03.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.392+0000 7fdffaa00640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdfe00b52e0 0x7fdfe00b77a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdffc1010d0 tx=0x7fdfe8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:03.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.393+0000 7fdff3e00640 1 -- 172.21.15.103:0/3177043438 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdfec0acc10 con 0x7fdffc0754c0 2024-09-11T00:21:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.594+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdffc02e5c0 con 0x7fdffc0754c0 2024-09-11T00:21:03.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.595+0000 7fdff3e00640 1 -- 172.21.15.103:0/3177043438 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+598 (secure 0 0 0) 0x7fdfec0b1640 con 0x7fdffc0754c0 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/489365644 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1436162684 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2651148903 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/817530946 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2651148903 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1510088401 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/477475422 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/477475422 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3506454988 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:03.597 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2690135380 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:03.598 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-11T00:21:03.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdfe00b52e0 msgr2=0x7fdfe00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:03.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdfe00b52e0 0x7fdfe00b77a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdffc1010d0 tx=0x7fdfe8005a20 comp rx=0 tx=0).stop 2024-09-11T00:21:03.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdffc0754c0 msgr2=0x7fdffc19b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdffc0754c0 0x7fdffc19b4d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00e9f0 tx=0x7fdfec00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 shutdown_connections 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdfe00b52e0 0x7fdfe00b77a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdffc10f3d0 0x7fdffc1a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdffc075de0 0x7fdffc19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 --2- 172.21.15.103:0/3177043438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdffc0754c0 0x7fdffc19b4d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 >> 172.21.15.103:0/3177043438 conn(0x7fdffc0fe830 msgr2=0x7fdffc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 shutdown_connections 2024-09-11T00:21:03.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:03.597+0000 7fe002800640 1 -- 172.21.15.103:0/3177043438 wait complete. 2024-09-11T00:21:03.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:03 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3177043438' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:21:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:03 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3177043438' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:21:04.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:03 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3177043438' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:21:04.449 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:21:04.449 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:21:04.486 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T00:21:04.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:04 smithi103 ceph-mon[30578]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:04.902 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:04 smithi117 ceph-mon[35525]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:05.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:04 smithi183 ceph-mon[35934]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:05.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 -- 172.21.15.103:0/1841113947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 msgr2=0x7f141c115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 --2- 172.21.15.103:0/1841113947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 0x7f141c115bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1414009ca0 tx=0x7f141402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 -- 172.21.15.103:0/1841113947 shutdown_connections 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 --2- 172.21.15.103:0/1841113947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 0x7f141c115bd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 --2- 172.21.15.103:0/1841113947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f141c077ff0 0x7f141c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 --2- 172.21.15.103:0/1841113947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c0776d0 0x7f141c077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 -- 172.21.15.103:0/1841113947 >> 172.21.15.103:0/1841113947 conn(0x7f141c100840 msgr2=0x7f141c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 -- 172.21.15.103:0/1841113947 shutdown_connections 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.473+0000 7f141b200640 1 -- 172.21.15.103:0/1841113947 wait complete. 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.474+0000 7f141b200640 1 Processor -- start 2024-09-11T00:21:05.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141b200640 1 -- start start 2024-09-11T00:21:05.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f141c0776d0 0x7f141c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:05.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c077ff0 0x7f141c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 0x7f141c1a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f141c119200 con 0x7f141c1137e0 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f141c119080 con 0x7f141c0776d0 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f141c119380 con 0x7f141c077ff0 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f1419400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c077ff0 0x7f141c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f141a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 0x7f141c1a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f1419400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c077ff0 0x7f141c19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:41050/0 (socket says 172.21.15.103:41050) 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f1419400640 1 -- 172.21.15.103:0/3884896242 learned_addr learned my addr 172.21.15.103:0/3884896242 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.475+0000 7f1419e00640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f141c0776d0 0x7f141c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.476+0000 7f1419400640 1 -- 172.21.15.103:0/3884896242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f141c0776d0 msgr2=0x7f141c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.476+0000 7f1419400640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f141c0776d0 0x7f141c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.476+0000 7f1419400640 1 -- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 msgr2=0x7f141c1a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:05.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.476+0000 7f1419400640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 0x7f141c1a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.476+0000 7f1419400640 1 -- 172.21.15.103:0/3884896242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141c1a56a0 con 0x7f141c077ff0 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.476+0000 7f1419400640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c077ff0 0x7f141c19fb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.476+0000 7f1412a00640 1 -- 172.21.15.103:0/3884896242 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140800cd80 con 0x7f141c077ff0 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.477+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141c1a5990 con 0x7f141c077ff0 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.477+0000 7f1412a00640 1 -- 172.21.15.103:0/3884896242 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14080108e0 con 0x7f141c077ff0 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.477+0000 7f1412a00640 1 -- 172.21.15.103:0/3884896242 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408018980 con 0x7f141c077ff0 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.477+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f141c1ac220 con 0x7f141c077ff0 2024-09-11T00:21:05.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.478+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141c078be0 con 0x7f141c077ff0 2024-09-11T00:21:05.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.481+0000 7f1412a00640 1 -- 172.21.15.103:0/3884896242 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f1408010430 con 0x7f141c077ff0 2024-09-11T00:21:05.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.482+0000 7f1412a00640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14000b5390 0x7f14000b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:05.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.482+0000 7f1412a00640 1 -- 172.21.15.103:0/3884896242 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f14080e65e0 con 0x7f141c077ff0 2024-09-11T00:21:05.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.482+0000 7f1419e00640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14000b5390 0x7f14000b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:05.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.483+0000 7f1412a00640 1 -- 172.21.15.103:0/3884896242 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f14080acbb0 con 0x7f141c077ff0 2024-09-11T00:21:05.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.483+0000 7f1419e00640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14000b5390 0x7f14000b7850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f140c009bd0 tx=0x7f140c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:05.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.682+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f141c077ab0 con 0x7f141c077ff0 2024-09-11T00:21:05.685 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.683+0000 7f1412a00640 1 -- 172.21.15.103:0/3884896242 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v85) ==== 81+0+598 (secure 0 0 0) 0x7f14080b15e0 con 0x7f141c077ff0 2024-09-11T00:21:05.685 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/489365644 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:05.685 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1436162684 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2651148903 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/817530946 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2651148903 2024-09-12T00:12:26.257361+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1510088401 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/477475422 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/477475422 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3506454988 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2690135380 2024-09-12T00:11:57.620215+0000 2024-09-11T00:21:05.686 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.685+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14000b5390 msgr2=0x7f14000b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14000b5390 0x7f14000b7850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f140c009bd0 tx=0x7f140c005f90 comp rx=0 tx=0).stop 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c077ff0 msgr2=0x7f141c19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c077ff0 0x7f141c19fb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 shutdown_connections 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14000b5390 0x7f14000b7850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f141c1137e0 0x7f141c1a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f141c077ff0 0x7f141c19fb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 --2- 172.21.15.103:0/3884896242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f141c0776d0 0x7f141c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:05.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 >> 172.21.15.103:0/3884896242 conn(0x7f141c100840 msgr2=0x7f141c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:05.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 shutdown_connections 2024-09-11T00:21:05.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:05.686+0000 7f141b200640 1 -- 172.21.15.103:0/3884896242 wait complete. 2024-09-11T00:21:06.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:05 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3884896242' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:21:06.082 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:05 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3884896242' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:21:06.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:05 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3884896242' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:21:06.497 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi103.front.sepia.ceph.com... 2024-09-11T00:21:06.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:21:06.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-11T00:21:06.552 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi103.stderr:ceph-fuse[76454]: fuse finished with error 0 and tester_r 0 2024-09-11T00:21:06.556 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T00:21:07.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:06 smithi183 ceph-mon[35934]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:07.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:06 smithi103 ceph-mon[30578]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:07.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:06 smithi117 ceph-mon[35525]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:08 smithi103 ceph-mon[30578]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:08 smithi183 ceph-mon[35934]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:08 smithi117 ceph-mon[35525]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-11T00:21:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:10 smithi183 ceph-mon[35934]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:10.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:10 smithi103 ceph-mon[30578]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:10 smithi117 ceph-mon[35525]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:12.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:21:12.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:21:12.659 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-11T00:21:12.705 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:21:12.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:12 smithi103 ceph-mon[30578]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:12.739 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:12 smithi117 ceph-mon[35525]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:12.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:12 smithi183 ceph-mon[35934]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:13.113 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 -- 172.21.15.103:0/1788360626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0108610 msgr2=0x7f6ab0108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 --2- 172.21.15.103:0/1788360626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0108610 0x7f6ab0108a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009900 tx=0x7f6a9c02a410 comp rx=0 tx=0).stop 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 -- 172.21.15.103:0/1788360626 shutdown_connections 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 --2- 172.21.15.103:0/1788360626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0108f90 0x7f6ab010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 --2- 172.21.15.103:0/1788360626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0108610 0x7f6ab0108a50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 --2- 172.21.15.103:0/1788360626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0102c00 0x7f6ab0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 -- 172.21.15.103:0/1788360626 >> 172.21.15.103:0/1788360626 conn(0x7f6ab00fe6d0 msgr2=0x7f6ab0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 -- 172.21.15.103:0/1788360626 shutdown_connections 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.794+0000 7f6ab5800640 1 -- 172.21.15.103:0/1788360626 wait complete. 2024-09-11T00:21:13.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.795+0000 7f6ab5800640 1 Processor -- start 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.795+0000 7f6ab5800640 1 -- start start 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.795+0000 7f6ab5800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0102c00 0x7f6ab00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.795+0000 7f6ab5800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0108610 0x7f6ab00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6ab5800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0108f90 0x7f6ab0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6ab5800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0112ca0 con 0x7f6ab0102c00 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6ab5800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0112b20 con 0x7f6ab0108f90 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6ab5800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab0112e20 con 0x7f6ab0108610 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0102c00 0x7f6ab00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aaf400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0108610 0x7f6ab00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0102c00 0x7f6ab00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55972/0 (socket says 172.21.15.103:55972) 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 -- 172.21.15.103:0/3598240900 learned_addr learned my addr 172.21.15.103:0/3598240900 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6ab4e00640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0108f90 0x7f6ab0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:13.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 -- 172.21.15.103:0/3598240900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0108610 msgr2=0x7f6ab00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0108610 0x7f6ab00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 -- 172.21.15.103:0/3598240900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0108f90 msgr2=0x7f6ab0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0108f90 0x7f6ab0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6aafe00640 1 -- 172.21.15.103:0/3598240900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab01a7430 con 0x7f6ab0102c00 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.796+0000 7f6ab4e00640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0108f90 0x7f6ab0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.797+0000 7f6aafe00640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0102c00 0x7f6ab00754a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000e9f0 tx=0x7f6aa000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.797+0000 7f6aacc00640 1 -- 172.21.15.103:0/3598240900 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000cd80 con 0x7f6ab0102c00 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.797+0000 7f6aacc00640 1 -- 172.21.15.103:0/3598240900 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6aa00054d0 con 0x7f6ab0102c00 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.797+0000 7f6aacc00640 1 -- 172.21.15.103:0/3598240900 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0010690 con 0x7f6ab0102c00 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.797+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab01a7720 con 0x7f6ab0102c00 2024-09-11T00:21:13.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.798+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab01a7be0 con 0x7f6ab0102c00 2024-09-11T00:21:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.799+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab0102fe0 con 0x7f6ab0102c00 2024-09-11T00:21:13.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.801+0000 7f6aacc00640 1 -- 172.21.15.103:0/3598240900 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f6aa0004e70 con 0x7f6ab0102c00 2024-09-11T00:21:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.802+0000 7f6aacc00640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6a940b5410 0x7f6a940b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:13.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.803+0000 7f6aacc00640 1 -- 172.21.15.103:0/3598240900 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f6aa00e6240 con 0x7f6ab0102c00 2024-09-11T00:21:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.803+0000 7f6aaf400640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6a940b5410 0x7f6a940b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:13.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.803+0000 7f6aaf400640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6a940b5410 0x7f6a940b78d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005ab0 tx=0x7f6a9c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:13.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.806+0000 7f6aacc00640 1 -- 172.21.15.103:0/3598240900 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6aa00acec0 con 0x7f6ab0102c00 2024-09-11T00:21:14.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:13.999+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ab01a7fc0 con 0x7f6ab0102c00 2024-09-11T00:21:14.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.001+0000 7f6aacc00640 1 -- 172.21.15.103:0/3598240900 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24244 (secure 0 0 0) 0x7f6aa00ac7c0 con 0x7f6ab0102c00 2024-09-11T00:21:14.003 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:14.004 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":85,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:20:41.548880+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:21:14.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.004+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6a940b5410 msgr2=0x7f6a940b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:14.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.004+0000 7f6ab5800640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6a940b5410 0x7f6a940b78d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005ab0 tx=0x7f6a9c005a20 comp rx=0 tx=0).stop 2024-09-11T00:21:14.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.004+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0102c00 msgr2=0x7f6ab00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:14.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.004+0000 7f6ab5800640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0102c00 0x7f6ab00754a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000e9f0 tx=0x7f6aa000edf0 comp rx=0 tx=0).stop 2024-09-11T00:21:14.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.004+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 shutdown_connections 2024-09-11T00:21:14.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.005+0000 7f6ab5800640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6a940b5410 0x7f6a940b78d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:14.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.005+0000 7f6ab5800640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ab0108f90 0x7f6ab0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:14.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.005+0000 7f6ab5800640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ab0108610 0x7f6ab00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:14.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.005+0000 7f6ab5800640 1 --2- 172.21.15.103:0/3598240900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ab0102c00 0x7f6ab00754a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:14.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.005+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 >> 172.21.15.103:0/3598240900 conn(0x7f6ab00fe6d0 msgr2=0x7f6ab0101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:14.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.005+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 shutdown_connections 2024-09-11T00:21:14.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:14.005+0000 7f6ab5800640 1 -- 172.21.15.103:0/3598240900 wait complete. 2024-09-11T00:21:14.260 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:14 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3598240900' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:14.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:14 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3598240900' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:14.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:14 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3598240900' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:14.843 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:21:15.224 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:15.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:15 smithi103 ceph-mon[30578]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:15.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:15 smithi117 ceph-mon[35525]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:15.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:15 smithi183 ceph-mon[35934]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.850+0000 7fe2b2a00640 1 -- 172.21.15.103:0/1937483372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac103cc0 msgr2=0x7fe2ac1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.850+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/1937483372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac103cc0 0x7fe2ac1040a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009ca0 tx=0x7fe29c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.851+0000 7fe2b2a00640 1 -- 172.21.15.103:0/1937483372 shutdown_connections 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.851+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/1937483372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac10d120 0x7fe2ac10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.851+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/1937483372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ac1045e0 0x7fe2ac10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.851+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/1937483372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac103cc0 0x7fe2ac1040a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.851+0000 7fe2b2a00640 1 -- 172.21.15.103:0/1937483372 >> 172.21.15.103:0/1937483372 conn(0x7fe2ac0fd700 msgr2=0x7fe2ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.851+0000 7fe2b2a00640 1 -- 172.21.15.103:0/1937483372 shutdown_connections 2024-09-11T00:21:15.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.851+0000 7fe2b2a00640 1 -- 172.21.15.103:0/1937483372 wait complete. 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 Processor -- start 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 -- start start 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ac103cc0 0x7fe2ac19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac1045e0 0x7fe2ac19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac10d120 0x7fe2ac1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac112d90 con 0x7fe2ac1045e0 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac112c10 con 0x7fe2ac103cc0 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2b2a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac112f10 con 0x7fe2ac10d120 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2ab400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac1045e0 0x7fe2ac19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.852+0000 7fe2ab400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac1045e0 0x7fe2ac19b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55982/0 (socket says 172.21.15.103:55982) 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2ab400640 1 -- 172.21.15.103:0/2605970527 learned_addr learned my addr 172.21.15.103:0/2605970527 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2b0a00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac10d120 0x7fe2ac1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2abe00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ac103cc0 0x7fe2ac19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:15.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2ab400640 1 -- 172.21.15.103:0/2605970527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac10d120 msgr2=0x7fe2ac1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:15.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2ab400640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac10d120 0x7fe2ac1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:15.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2ab400640 1 -- 172.21.15.103:0/2605970527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ac103cc0 msgr2=0x7fe2ac19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:15.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2ab400640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ac103cc0 0x7fe2ac19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:15.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2ab400640 1 -- 172.21.15.103:0/2605970527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac1a1520 con 0x7fe2ac1045e0 2024-09-11T00:21:15.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2abe00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ac103cc0 0x7fe2ac19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:15.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.853+0000 7fe2ab400640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac1045e0 0x7fe2ac19b9b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9c0 tx=0x7fe29800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:15.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.854+0000 7fe2a8c00640 1 -- 172.21.15.103:0/2605970527 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800ccc0 con 0x7fe2ac1045e0 2024-09-11T00:21:15.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.854+0000 7fe2a8c00640 1 -- 172.21.15.103:0/2605970527 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2980035f0 con 0x7fe2ac1045e0 2024-09-11T00:21:15.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.854+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac1a1810 con 0x7fe2ac1045e0 2024-09-11T00:21:15.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.854+0000 7fe2a8c00640 1 -- 172.21.15.103:0/2605970527 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800ce60 con 0x7fe2ac1045e0 2024-09-11T00:21:15.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.854+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac1a7ed0 con 0x7fe2ac1045e0 2024-09-11T00:21:15.857 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.855+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ac1040a0 con 0x7fe2ac1045e0 2024-09-11T00:21:15.859 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.858+0000 7fe2a8c00640 1 -- 172.21.15.103:0/2605970527 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fe298004000 con 0x7fe2ac1045e0 2024-09-11T00:21:15.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.859+0000 7fe2a8c00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2900b5360 0x7fe2900b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:15.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.859+0000 7fe2a8c00640 1 -- 172.21.15.103:0/2605970527 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fe2980e6090 con 0x7fe2ac1045e0 2024-09-11T00:21:15.861 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.859+0000 7fe2abe00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2900b5360 0x7fe2900b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:15.862 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.860+0000 7fe2abe00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2900b5360 0x7fe2900b7820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009430 tx=0x7fe29c009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:15.865 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:15.863+0000 7fe2a8c00640 1 -- 172.21.15.103:0/2605970527 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe2980accb0 con 0x7fe2ac1045e0 2024-09-11T00:21:16.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.052+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2ac19c5a0 con 0x7fe2ac1045e0 2024-09-11T00:21:16.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.054+0000 7fe2a8c00640 1 -- 172.21.15.103:0/2605970527 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24244 (secure 0 0 0) 0x7fe2980ac5b0 con 0x7fe2ac1045e0 2024-09-11T00:21:16.056 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:16.056 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":85,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:20:41.548880+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.056+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2900b5360 msgr2=0x7fe2900b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.056+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2900b5360 0x7fe2900b7820 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009430 tx=0x7fe29c009360 comp rx=0 tx=0).stop 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.056+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac1045e0 msgr2=0x7fe2ac19b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.056+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac1045e0 0x7fe2ac19b9b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9c0 tx=0x7fe29800ee80 comp rx=0 tx=0).stop 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 shutdown_connections 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2900b5360 0x7fe2900b7820 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2ac10d120 0x7fe2ac1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2ac1045e0 0x7fe2ac19b9b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 --2- 172.21.15.103:0/2605970527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2ac103cc0 0x7fe2ac19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 >> 172.21.15.103:0/2605970527 conn(0x7fe2ac0fd700 msgr2=0x7fe2ac10f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:16.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 shutdown_connections 2024-09-11T00:21:16.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:16.057+0000 7fe2b2a00640 1 -- 172.21.15.103:0/2605970527 wait complete. 2024-09-11T00:21:16.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:16 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2605970527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:16.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:16 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2605970527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:16.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:16 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2605970527' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:16.992 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T00:21:17.364 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:17.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:17 smithi103 ceph-mon[30578]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:17.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:17 smithi117 ceph-mon[35525]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:17.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:17 smithi183 ceph-mon[35934]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:17.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 -- 172.21.15.103:0/2391551123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c10f200 msgr2=0x7f509c1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:17.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 --2- 172.21.15.103:0/2391551123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c10f200 0x7f509c1115f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5094009900 tx=0x7f509402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 -- 172.21.15.103:0/2391551123 shutdown_connections 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 --2- 172.21.15.103:0/2391551123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c10f200 0x7f509c1115f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 --2- 172.21.15.103:0/2391551123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c075e00 0x7f509c10ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 --2- 172.21.15.103:0/2391551123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f509c0754e0 0x7f509c0758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 -- 172.21.15.103:0/2391551123 >> 172.21.15.103:0/2391551123 conn(0x7f509c0fe660 msgr2=0x7f509c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 -- 172.21.15.103:0/2391551123 shutdown_connections 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.919+0000 7f50a1200640 1 -- 172.21.15.103:0/2391551123 wait complete. 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.920+0000 7f50a1200640 1 Processor -- start 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.920+0000 7f50a1200640 1 -- start start 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f50a1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f509c0754e0 0x7f509c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f50a1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c075e00 0x7f509c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:17.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f50a1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c10f200 0x7f509c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f50a1200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f509c114de0 con 0x7f509c075e00 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f50a1200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f509c114c60 con 0x7f509c10f200 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f50a1200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f509c114f60 con 0x7f509c0754e0 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c075e00 0x7f509c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c075e00 0x7f509c19b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55998/0 (socket says 172.21.15.103:55998) 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 -- 172.21.15.103:0/2861262553 learned_addr learned my addr 172.21.15.103:0/2861262553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f509b400640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f509c0754e0 0x7f509c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f509be00640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c10f200 0x7f509c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 -- 172.21.15.103:0/2861262553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f509c0754e0 msgr2=0x7f509c19b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f509c0754e0 0x7f509c19b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 -- 172.21.15.103:0/2861262553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c10f200 msgr2=0x7f509c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c10f200 0x7f509c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:17.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f508be00640 1 -- 172.21.15.103:0/2861262553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c1a12a0 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.921+0000 7f509be00640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c10f200 0x7f509c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.922+0000 7f508be00640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c075e00 0x7f509c19b730 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f508400ea60 tx=0x7f508400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.922+0000 7f5098c00640 1 -- 172.21.15.103:0/2861262553 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508400cde0 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.922+0000 7f5098c00640 1 -- 172.21.15.103:0/2861262553 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50840035f0 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.922+0000 7f5098c00640 1 -- 172.21.15.103:0/2861262553 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084010690 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.922+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509c1a1590 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.922+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509c1a7c50 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.923+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509c0758c0 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.926+0000 7f5098c00640 1 -- 172.21.15.103:0/2861262553 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f5084004000 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.927+0000 7f5098c00640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f507c0b5390 0x7f507c0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.928+0000 7f5098c00640 1 -- 172.21.15.103:0/2861262553 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f50840e6200 con 0x7f509c075e00 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.928+0000 7f509b400640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f507c0b5390 0x7f507c0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:17.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.929+0000 7f509b400640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f507c0b5390 0x7f507c0b7850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f508c005fd0 tx=0x7f508c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:17.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:17.932+0000 7f5098c00640 1 -- 172.21.15.103:0/2861262553 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f50840ace20 con 0x7f509c075e00 2024-09-11T00:21:18.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:18.153+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f509c1a8000 con 0x7f509c075e00 2024-09-11T00:21:19.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.083+0000 7f5098c00640 1 -- 172.21.15.103:0/2861262553 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f50840a6e20 con 0x7f509c075e00 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f507c0b5390 msgr2=0x7f507c0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f507c0b5390 0x7f507c0b7850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f508c005fd0 tx=0x7f508c009500 comp rx=0 tx=0).stop 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c075e00 msgr2=0x7f509c19b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c075e00 0x7f509c19b730 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f508400ea60 tx=0x7f508400ef20 comp rx=0 tx=0).stop 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 shutdown_connections 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f507c0b5390 0x7f507c0b7850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f509c10f200 0x7f509c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:19.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f509c075e00 0x7f509c19b730 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:19.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 --2- 172.21.15.103:0/2861262553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f509c0754e0 0x7f509c19b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:19.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 >> 172.21.15.103:0/2861262553 conn(0x7f509c0fe660 msgr2=0x7f509c111370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:19.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 shutdown_connections 2024-09-11T00:21:19.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:19.087+0000 7f50a1200640 1 -- 172.21.15.103:0/2861262553 wait complete. 2024-09-11T00:21:19.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:19 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2861262553' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:21:19.451 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:19 smithi103 ceph-mon[30578]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:19.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:19 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2861262553' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:21:19.452 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:19 smithi117 ceph-mon[35525]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:19.541 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:19 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2861262553' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-11T00:21:19.541 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:19 smithi183 ceph-mon[35934]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:19.947 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-11T00:21:19.947 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-11T00:21:20.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:20 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2861262553' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:21:20.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:20 smithi103 ceph-mon[30578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:21:20.327 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:20 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2861262553' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:21:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:20 smithi117 ceph-mon[35525]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:21:20.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:20 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2861262553' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T00:21:20.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:20 smithi183 ceph-mon[35934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 -- 172.21.15.103:0/1050504185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 msgr2=0x7f9030105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 --2- 172.21.15.103:0/1050504185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f9030105380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9020009ca0 tx=0x7f902002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 -- 172.21.15.103:0/1050504185 shutdown_connections 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 --2- 172.21.15.103:0/1050504185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f903010b330 0x7f9030111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 --2- 172.21.15.103:0/1050504185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903010a9b0 0x7f903010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 --2- 172.21.15.103:0/1050504185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f9030105380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 -- 172.21.15.103:0/1050504185 >> 172.21.15.103:0/1050504185 conn(0x7f9030100a50 msgr2=0x7f9030102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 -- 172.21.15.103:0/1050504185 shutdown_connections 2024-09-11T00:21:20.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.969+0000 7f9035c00640 1 -- 172.21.15.103:0/1050504185 wait complete. 2024-09-11T00:21:20.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 Processor -- start 2024-09-11T00:21:20.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 -- start start 2024-09-11T00:21:20.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f903019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903010a9b0 0x7f903019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f903010b330 0x7f90301a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9030114f60 con 0x7f903010b330 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9030114de0 con 0x7f9030104fa0 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.970+0000 7f9035c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90301150e0 con 0x7f903010a9b0 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f903019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f903010b330 0x7f90301a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f903019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45918/0 (socket says 172.21.15.103:45918) 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 -- 172.21.15.103:0/1302230339 learned_addr learned my addr 172.21.15.103:0/1302230339 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:20.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f9026000640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903010a9b0 0x7f903019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:20.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 -- 172.21.15.103:0/1302230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903010a9b0 msgr2=0x7f903019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:20.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903010a9b0 0x7f903019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:20.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 -- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f903010b330 msgr2=0x7f90301a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:20.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f903010b330 0x7f90301a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 -- 172.21.15.103:0/1302230339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90301a15c0 con 0x7f9030104fa0 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.971+0000 7f902ea00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f903019b510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9020033040 tx=0x7f90200316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.972+0000 7f9027e00640 1 -- 172.21.15.103:0/1302230339 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020003b70 con 0x7f9030104fa0 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.972+0000 7f9027e00640 1 -- 172.21.15.103:0/1302230339 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9020003d10 con 0x7f9030104fa0 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.972+0000 7f9027e00640 1 -- 172.21.15.103:0/1302230339 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902002f9e0 con 0x7f9030104fa0 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.972+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90301a1850 con 0x7f9030104fa0 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.972+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90301a7f70 con 0x7f9030104fa0 2024-09-11T00:21:20.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.973+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9030105380 con 0x7f9030104fa0 2024-09-11T00:21:20.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.975+0000 7f9027e00640 1 -- 172.21.15.103:0/1302230339 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f9020004b40 con 0x7f9030104fa0 2024-09-11T00:21:20.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.976+0000 7f9027e00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90140b5360 0x7f90140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:20.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.977+0000 7f9027e00640 1 -- 172.21.15.103:0/1302230339 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f9020107120 con 0x7f9030104fa0 2024-09-11T00:21:20.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.977+0000 7f9026000640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90140b5360 0x7f90140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:20.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.977+0000 7f9026000640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90140b5360 0x7f90140b7820 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9030108410 tx=0x7f9018005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:20.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:20.978+0000 7f9027e00640 1 -- 172.21.15.103:0/1302230339 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f90200cdd40 con 0x7f9030104fa0 2024-09-11T00:21:21.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:21 smithi103 ceph-mon[30578]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:21.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:21.221+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f903002e5c0 con 0x7f9030104fa0 2024-09-11T00:21:21.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:21 smithi117 ceph-mon[35525]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:21.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:21 smithi183 ceph-mon[35934]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-11T00:21:22.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.101+0000 7f9027e00640 1 -- 172.21.15.103:0/1302230339 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f90200cd640 con 0x7f9030104fa0 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.103+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90140b5360 msgr2=0x7f90140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.103+0000 7f9035c00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90140b5360 0x7f90140b7820 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9030108410 tx=0x7f9018005f90 comp rx=0 tx=0).stop 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.103+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 msgr2=0x7f903019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.103+0000 7f9035c00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f903019b510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9020033040 tx=0x7f90200316f0 comp rx=0 tx=0).stop 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 shutdown_connections 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f90140b5360 0x7f90140b7820 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f903010b330 0x7f90301a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f903010a9b0 0x7f903019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 --2- 172.21.15.103:0/1302230339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9030104fa0 0x7f903019b510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 >> 172.21.15.103:0/1302230339 conn(0x7f9030100a50 msgr2=0x7f9030102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:22.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 shutdown_connections 2024-09-11T00:21:22.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:22.104+0000 7f9035c00640 1 -- 172.21.15.103:0/1302230339 wait complete. 2024-09-11T00:21:22.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:22 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:21:22.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:22 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1302230339' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-11T00:21:22.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:22 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-11T00:21:22.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:22 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:21:22.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:22 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1302230339' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-11T00:21:22.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:22 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-11T00:21:22.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:22 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:21:22.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:22 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1302230339' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-11T00:21:22.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:22 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-09-11T00:21:23.361 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:21:23.361 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T00:21:23.361 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:21:23.361 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:21:23.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T00:21:23.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T00:21:23.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:23.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.362 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:23 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T00:21:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:21:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T00:21:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:21:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:21:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T00:21:23.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T00:21:23.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:23.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:23 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:23 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:23.802 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:24.338 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:24 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] up:active 2024-09-11T00:21:24.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:24 smithi103 ceph-mon[30578]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:21:24.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:24 smithi103 ceph-mon[30578]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:21:24.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:24 smithi103 ceph-mon[30578]: Cluster is now healthy 2024-09-11T00:21:24.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:24 smithi103 ceph-mon[30578]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T00:21:24.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:24 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T00:21:24.339 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:24 smithi103 ceph-mon[30578]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T00:21:24.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: 2024-09-11T00:21:24.142+0000 7f56541e2640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T00:21:24.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: 2024-09-11T00:21:24.142+0000 7f56541e2640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-11T00:21:24.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] up:active 2024-09-11T00:21:24.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-mon[35525]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:21:24.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-mon[35525]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:21:24.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-mon[35525]: Cluster is now healthy 2024-09-11T00:21:24.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-mon[35525]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T00:21:24.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T00:21:24.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:24 smithi117 ceph-mon[35525]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T00:21:24.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 -- 172.21.15.103:0/1324238132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 msgr2=0x7f2a4410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:24.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1324238132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a4410f950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009ca0 tx=0x7f2a3c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:24.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 -- 172.21.15.103:0/1324238132 shutdown_connections 2024-09-11T00:21:24.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1324238132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a4410f950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1324238132 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a441087d0 0x7f2a44108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1324238132 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44102dc0 0x7f2a441031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 -- 172.21.15.103:0/1324238132 >> 172.21.15.103:0/1324238132 conn(0x7f2a440fe830 msgr2=0x7f2a44100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.468+0000 7f2a4a800640 1 -- 172.21.15.103:0/1324238132 shutdown_connections 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.469+0000 7f2a4a800640 1 -- 172.21.15.103:0/1324238132 wait complete. 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.469+0000 7f2a4a800640 1 Processor -- start 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a4a800640 1 -- start start 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a4a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44102dc0 0x7f2a440754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a4a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a441087d0 0x7f2a440759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a4a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a44075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a4a800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44112dd0 con 0x7f2a44109150 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a4a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44112c50 con 0x7f2a44102dc0 2024-09-11T00:21:24.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a4a800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44112f50 con 0x7f2a441087d0 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a43e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a44075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a43400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44102dc0 0x7f2a440754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a42a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a441087d0 0x7f2a440759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a43e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a44075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50568/0 (socket says 172.21.15.103:50568) 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.470+0000 7f2a43e00640 1 -- 172.21.15.103:0/1320235013 learned_addr learned my addr 172.21.15.103:0/1320235013 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a43400640 1 -- 172.21.15.103:0/1320235013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a441087d0 msgr2=0x7f2a440759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a43400640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a441087d0 0x7f2a440759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a43400640 1 -- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 msgr2=0x7f2a44075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a43400640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a44075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a43400640 1 -- 172.21.15.103:0/1320235013 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a441a76a0 con 0x7f2a44102dc0 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a43e00640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a44075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:24.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a43400640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44102dc0 0x7f2a440754a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9f0 tx=0x7f2a3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:24.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.471+0000 7f2a3be00640 1 -- 172.21.15.103:0/1320235013 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3400cd80 con 0x7f2a44102dc0 2024-09-11T00:21:24.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.472+0000 7f2a3be00640 1 -- 172.21.15.103:0/1320235013 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a340035f0 con 0x7f2a44102dc0 2024-09-11T00:21:24.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.472+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a441a7990 con 0x7f2a44102dc0 2024-09-11T00:21:24.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.472+0000 7f2a3be00640 1 -- 172.21.15.103:0/1320235013 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34010670 con 0x7f2a44102dc0 2024-09-11T00:21:24.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.472+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a441a7f20 con 0x7f2a44102dc0 2024-09-11T00:21:24.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.473+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a441031a0 con 0x7f2a44102dc0 2024-09-11T00:21:24.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.477+0000 7f2a3be00640 1 -- 172.21.15.103:0/1320235013 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f2a34004000 con 0x7f2a44102dc0 2024-09-11T00:21:24.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.478+0000 7f2a3be00640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2a280b5360 0x7f2a280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:24.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.478+0000 7f2a3be00640 1 -- 172.21.15.103:0/1320235013 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f2a340e5fe0 con 0x7f2a44102dc0 2024-09-11T00:21:24.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.479+0000 7f2a42a00640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2a280b5360 0x7f2a280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:24.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.479+0000 7f2a42a00640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2a280b5360 0x7f2a280b7820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a44076f90 tx=0x7f2a30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:24.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.482+0000 7f2a3be00640 1 -- 172.21.15.103:0/1320235013 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2a340acc60 con 0x7f2a44102dc0 2024-09-11T00:21:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:24 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] up:active 2024-09-11T00:21:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:24 smithi183 ceph-mon[35934]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T00:21:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:24 smithi183 ceph-mon[35934]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T00:21:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:24 smithi183 ceph-mon[35934]: Cluster is now healthy 2024-09-11T00:21:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:24 smithi183 ceph-mon[35934]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-11T00:21:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:24 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-11T00:21:24.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:24 smithi183 ceph-mon[35934]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-11T00:21:24.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:24.751+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2a440767a0 con 0x7f2a44102dc0 2024-09-11T00:21:25.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.204+0000 7f2a3be00640 1 -- 172.21.15.103:0/1320235013 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f2a340ac560 con 0x7f2a44102dc0 2024-09-11T00:21:25.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2a280b5360 msgr2=0x7f2a280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2a280b5360 0x7f2a280b7820 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a44076f90 tx=0x7f2a30005f90 comp rx=0 tx=0).stop 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44102dc0 msgr2=0x7f2a440754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44102dc0 0x7f2a440754a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400e9f0 tx=0x7f2a3400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 shutdown_connections 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2a280b5360 0x7f2a280b7820 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a44109150 0x7f2a44075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a441087d0 0x7f2a440759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 --2- 172.21.15.103:0/1320235013 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2a44102dc0 0x7f2a440754a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 >> 172.21.15.103:0/1320235013 conn(0x7f2a440fe830 msgr2=0x7f2a44100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 shutdown_connections 2024-09-11T00:21:25.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:25.207+0000 7f2a4a800640 1 -- 172.21.15.103:0/1320235013 wait complete. 2024-09-11T00:21:25.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] up:active 2024-09-11T00:21:25.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:21:25.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: pgmap v344: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-09-11T00:21:25.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1320235013' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:21:25.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:21:25.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:21:25.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:21:25.421 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] up:active 2024-09-11T00:21:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:21:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: pgmap v344: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-09-11T00:21:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1320235013' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:21:25.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:21:25.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:21:25.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:21:25.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] up:active 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: pgmap v344: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 4 op/s 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1320235013' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:21:25.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:21:26.020 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T00:21:26.395 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:26.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:26 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:21:26.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:26 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:21:26.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:26 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:21:26.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:26 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:21:26.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:26 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:21:26.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:26 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:21:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:26 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T00:21:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:26 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-11T00:21:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:26 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:21:27.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 -- 172.21.15.103:0/1720850136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e4104fc0 msgr2=0x7fa9e41053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:27.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 --2- 172.21.15.103:0/1720850136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e4104fc0 0x7fa9e41053a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009ca0 tx=0x7fa9d402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:27.008 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 -- 172.21.15.103:0/1720850136 shutdown_connections 2024-09-11T00:21:27.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 --2- 172.21.15.103:0/1720850136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e410b350 0x7fa9e4111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:27.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 --2- 172.21.15.103:0/1720850136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9e410a9d0 0x7fa9e410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:27.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 --2- 172.21.15.103:0/1720850136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e4104fc0 0x7fa9e41053a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:27.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 -- 172.21.15.103:0/1720850136 >> 172.21.15.103:0/1720850136 conn(0x7fa9e41009f0 msgr2=0x7fa9e4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:27.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 -- 172.21.15.103:0/1720850136 shutdown_connections 2024-09-11T00:21:27.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.006+0000 7fa9eb000640 1 -- 172.21.15.103:0/1720850136 wait complete. 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.007+0000 7fa9eb000640 1 Processor -- start 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.007+0000 7fa9eb000640 1 -- start start 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9eb000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4104fc0 0x7fa9e419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9eb000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e410a9d0 0x7fa9e419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9eb000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9e410b350 0x7fa9e41a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9eb000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4114f80 con 0x7fa9e410a9d0 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9eb000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4114e00 con 0x7fa9e4104fc0 2024-09-11T00:21:27.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9eb000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e4115100 con 0x7fa9e410b350 2024-09-11T00:21:27.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e410a9d0 0x7fa9e419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:27.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9e9000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9e410b350 0x7fa9e41a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:27.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9db400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e410a9d0 0x7fa9e419b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50588/0 (socket says 172.21.15.103:50588) 2024-09-11T00:21:27.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9db400640 1 -- 172.21.15.103:0/2565879573 learned_addr learned my addr 172.21.15.103:0/2565879573 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:27.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.008+0000 7fa9e3e00640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4104fc0 0x7fa9e419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9db400640 1 -- 172.21.15.103:0/2565879573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9e410b350 msgr2=0x7fa9e41a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9db400640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9e410b350 0x7fa9e41a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9db400640 1 -- 172.21.15.103:0/2565879573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4104fc0 msgr2=0x7fa9e419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9db400640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4104fc0 0x7fa9e419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9db400640 1 -- 172.21.15.103:0/2565879573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e41a1540 con 0x7fa9e410a9d0 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9db400640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e410a9d0 0x7fa9e419b9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e9f0 tx=0x7fa9cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9e1600640 1 -- 172.21.15.103:0/2565879573 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00cd80 con 0x7fa9e410a9d0 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9e1600640 1 -- 172.21.15.103:0/2565879573 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc0054d0 con 0x7fa9e410a9d0 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e41a1830 con 0x7fa9e410a9d0 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9e1600640 1 -- 172.21.15.103:0/2565879573 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc010690 con 0x7fa9e410a9d0 2024-09-11T00:21:27.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.009+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e41a7ef0 con 0x7fa9e410a9d0 2024-09-11T00:21:27.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.011+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e41053a0 con 0x7fa9e410a9d0 2024-09-11T00:21:27.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.013+0000 7fa9e1600640 1 -- 172.21.15.103:0/2565879573 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fa9cc004e90 con 0x7fa9e410a9d0 2024-09-11T00:21:27.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.014+0000 7fa9e1600640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa9c80b52e0 0x7fa9c80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:27.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.014+0000 7fa9e1600640 1 -- 172.21.15.103:0/2565879573 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fa9cc0e6280 con 0x7fa9e410a9d0 2024-09-11T00:21:27.016 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.014+0000 7fa9e3e00640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa9c80b52e0 0x7fa9c80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:27.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.015+0000 7fa9e3e00640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa9c80b52e0 0x7fa9c80b77a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009430 tx=0x7fa9d4009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:27.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.018+0000 7fa9e1600640 1 -- 172.21.15.103:0/2565879573 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa9cc0acea0 con 0x7fa9e410a9d0 2024-09-11T00:21:27.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:27.245+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa9e410d160 con 0x7fa9e410a9d0 2024-09-11T00:21:27.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]} : dispatch 2024-09-11T00:21:27.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:27 smithi183 ceph-mon[35934]: pgmap v345: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-11T00:21:27.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]} : dispatch 2024-09-11T00:21:27.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:27 smithi103 ceph-mon[30578]: pgmap v345: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-11T00:21:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]} : dispatch 2024-09-11T00:21:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-09-11T00:21:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-11T00:21:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2024-09-11T00:21:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]} : dispatch 2024-09-11T00:21:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2024-09-11T00:21:27.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]} : dispatch 2024-09-11T00:21:27.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]} : dispatch 2024-09-11T00:21:27.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]} : dispatch 2024-09-11T00:21:27.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]} : dispatch 2024-09-11T00:21:27.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:27 smithi117 ceph-mon[35525]: pgmap v345: 129 pgs: 129 active+clean; 583 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-09-11T00:21:28.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.211+0000 7fa9e1600640 1 -- 172.21.15.103:0/2565879573 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fa9cc0ac7a0 con 0x7fa9e410a9d0 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa9c80b52e0 msgr2=0x7fa9c80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa9c80b52e0 0x7fa9c80b77a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4009430 tx=0x7fa9d4009360 comp rx=0 tx=0).stop 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e410a9d0 msgr2=0x7fa9e419b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e410a9d0 0x7fa9e419b9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc00e9f0 tx=0x7fa9cc00edf0 comp rx=0 tx=0).stop 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 shutdown_connections 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa9c80b52e0 0x7fa9c80b77a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa9e410b350 0x7fa9e41a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9e410a9d0 0x7fa9e419b9d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:28.216 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 --2- 172.21.15.103:0/2565879573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e4104fc0 0x7fa9e419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:28.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 >> 172.21.15.103:0/2565879573 conn(0x7fa9e41009f0 msgr2=0x7fa9e4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:28.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 shutdown_connections 2024-09-11T00:21:28.217 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:28.214+0000 7fa9eb000640 1 -- 172.21.15.103:0/2565879573 wait complete. 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-11T00:21:28.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-11T00:21:28.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T00:21:28.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:28 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2565879573' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-11T00:21:28.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-11T00:21:28.619 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T00:21:28.619 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:28 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2565879573' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:21:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-09-11T00:21:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-11T00:21:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [5, 8]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T00:21:28.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:28 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2565879573' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-11T00:21:29.122 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-11T00:21:29.122 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T00:21:29.499 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:29 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2565879573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:21:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:29 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:21:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:29 smithi183 ceph-mon[35934]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T00:21:29.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:29 smithi183 ceph-mon[35934]: pgmap v348: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-11T00:21:29.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:29 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2565879573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:21:29.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:29 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:21:29.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:29 smithi103 ceph-mon[30578]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T00:21:29.638 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:29 smithi103 ceph-mon[30578]: pgmap v348: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-11T00:21:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:29 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2565879573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T00:21:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:29 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:21:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:29 smithi117 ceph-mon[35525]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T00:21:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:29 smithi117 ceph-mon[35525]: pgmap v348: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-11T00:21:30.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 -- 172.21.15.103:0/3821603821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868102c00 msgr2=0x7f9868102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:30.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 --2- 172.21.15.103:0/3821603821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868102c00 0x7f9868102fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9858009900 tx=0x7f985802a410 comp rx=0 tx=0).stop 2024-09-11T00:21:30.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 -- 172.21.15.103:0/3821603821 shutdown_connections 2024-09-11T00:21:30.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 --2- 172.21.15.103:0/3821603821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868108f90 0x7f986810f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 --2- 172.21.15.103:0/3821603821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9868108610 0x7f9868108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 --2- 172.21.15.103:0/3821603821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868102c00 0x7f9868102fe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 -- 172.21.15.103:0/3821603821 >> 172.21.15.103:0/3821603821 conn(0x7f98680fe6d0 msgr2=0x7f9868100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:30.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 -- 172.21.15.103:0/3821603821 shutdown_connections 2024-09-11T00:21:30.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.233+0000 7f9867200640 1 -- 172.21.15.103:0/3821603821 wait complete. 2024-09-11T00:21:30.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 Processor -- start 2024-09-11T00:21:30.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 -- start start 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9868102c00 0x7f986819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868108610 0x7f986819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868108f90 0x7f98681a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112ca0 con 0x7f9868102c00 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112b20 con 0x7f9868108610 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.234+0000 7f9867200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9868112e20 con 0x7f9868108f90 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868108610 0x7f986819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868108610 0x7f986819b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:54592/0 (socket says 172.21.15.103:54592) 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 -- 172.21.15.103:0/2532393969 learned_addr learned my addr 172.21.15.103:0/2532393969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:30.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9866800640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868108f90 0x7f98681a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865e00640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9868102c00 0x7f986819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 -- 172.21.15.103:0/2532393969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868108f90 msgr2=0x7f98681a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868108f90 0x7f98681a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 -- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9868102c00 msgr2=0x7f986819b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9868102c00 0x7f986819b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 -- 172.21.15.103:0/2532393969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98681a1280 con 0x7f9868108610 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865e00640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9868102c00 0x7f986819b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.235+0000 7f9865400640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868108610 0x7f986819b710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f985400e9f0 tx=0x7f985400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:30.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.236+0000 7f985ea00640 1 -- 172.21.15.103:0/2532393969 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985400cd80 con 0x7f9868108610 2024-09-11T00:21:30.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.236+0000 7f985ea00640 1 -- 172.21.15.103:0/2532393969 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98540108e0 con 0x7f9868108610 2024-09-11T00:21:30.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.236+0000 7f985ea00640 1 -- 172.21.15.103:0/2532393969 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854018980 con 0x7f9868108610 2024-09-11T00:21:30.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.236+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681a1570 con 0x7f9868108610 2024-09-11T00:21:30.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.236+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a7e00 con 0x7f9868108610 2024-09-11T00:21:30.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.237+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9868103eb0 con 0x7f9868108610 2024-09-11T00:21:30.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.239+0000 7f985ea00640 1 -- 172.21.15.103:0/2532393969 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7f9854010430 con 0x7f9868108610 2024-09-11T00:21:30.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.241+0000 7f985ea00640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f984c0b5410 0x7f984c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:30.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.241+0000 7f985ea00640 1 -- 172.21.15.103:0/2532393969 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f98540e62a0 con 0x7f9868108610 2024-09-11T00:21:30.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.241+0000 7f9865e00640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f984c0b5410 0x7f984c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:30.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.242+0000 7f9865e00640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f984c0b5410 0x7f984c0b78d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9858000c00 tx=0x7f9858005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:30.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.244+0000 7f985ea00640 1 -- 172.21.15.103:0/2532393969 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f98540acda0 con 0x7f9868108610 2024-09-11T00:21:30.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.456+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9868063590 con 0x7f984c0b5410 2024-09-11T00:21:30.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.485+0000 7f985ea00640 1 -- 172.21.15.103:0/2532393969 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9868063590 con 0x7f984c0b5410 2024-09-11T00:21:30.488 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:30 smithi103 ceph-mon[30578]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-11T00:21:30.488 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:21:30 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:21:30.463+0000 7f7869200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:21:30.488 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:21:30 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:21:30.463+0000 7f7869200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:21:30.488 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:21:30 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:21:30.463+0000 7f7869200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:21:30.488 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:21:30 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:21:30.463+0000 7f7869200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:21:30.488 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:21:30 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:21:30.463+0000 7f7869200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f984c0b5410 msgr2=0x7f984c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f984c0b5410 0x7f984c0b78d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9858000c00 tx=0x7f9858005a20 comp rx=0 tx=0).stop 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868108610 msgr2=0x7f986819b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868108610 0x7f986819b710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f985400e9f0 tx=0x7f985400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 shutdown_connections 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f984c0b5410 0x7f984c0b78d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868108f90 0x7f98681a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9868108610 0x7f986819b710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 --2- 172.21.15.103:0/2532393969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9868102c00 0x7f986819b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:30.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 >> 172.21.15.103:0/2532393969 conn(0x7f98680fe6d0 msgr2=0x7f9868104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:30.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 shutdown_connections 2024-09-11T00:21:30.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:30.488+0000 7f9867200640 1 -- 172.21.15.103:0/2532393969 wait complete. 2024-09-11T00:21:30.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:30 smithi183 ceph-mon[35934]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-11T00:21:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:30 smithi117 ceph-mon[35525]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-11T00:21:31.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T00:21:31.418 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:31 smithi103 ceph-mon[30578]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:31.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:31 smithi183 ceph-mon[35934]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:31.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:31 smithi117 ceph-mon[35525]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 3.0 KiB/s wr, 14 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:31.761 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:32.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 -- 172.21.15.103:0/663083329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa284075f40 msgr2=0x7fa28410ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:32.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 --2- 172.21.15.103:0/663083329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa284075f40 0x7fa28410ee10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa270009900 tx=0x7fa27002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 -- 172.21.15.103:0/663083329 shutdown_connections 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 --2- 172.21.15.103:0/663083329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa28410f3e0 0x7fa2841117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 --2- 172.21.15.103:0/663083329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa284075f40 0x7fa28410ee10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 --2- 172.21.15.103:0/663083329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa284075540 0x7fa284075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 -- 172.21.15.103:0/663083329 >> 172.21.15.103:0/663083329 conn(0x7fa2840fe830 msgr2=0x7fa284100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 -- 172.21.15.103:0/663083329 shutdown_connections 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.426+0000 7fa28b200640 1 -- 172.21.15.103:0/663083329 wait complete. 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 Processor -- start 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 -- start start 2024-09-11T00:21:32.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa284075540 0x7fa28419b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa284075f40 0x7fa28419ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa28410f3e0 0x7fa2841a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa284114fe0 con 0x7fa28410f3e0 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa284114e60 con 0x7fa284075540 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.427+0000 7fa28b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa284115160 con 0x7fa284075f40 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa289200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa28410f3e0 0x7fa2841a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa284075540 0x7fa28419b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa284075540 0x7fa28419b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33010/0 (socket says 172.21.15.103:33010) 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa289200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa28410f3e0 0x7fa2841a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36826/0 (socket says 172.21.15.103:36826) 2024-09-11T00:21:32.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 -- 172.21.15.103:0/1493778669 learned_addr learned my addr 172.21.15.103:0/1493778669 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283400640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa284075f40 0x7fa28419ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 -- 172.21.15.103:0/1493778669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa284075f40 msgr2=0x7fa28419ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa284075f40 0x7fa28419ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 -- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa28410f3e0 msgr2=0x7fa2841a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa28410f3e0 0x7fa2841a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa283e00640 1 -- 172.21.15.103:0/1493778669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2841a15a0 con 0x7fa284075540 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.428+0000 7fa289200640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa28410f3e0 0x7fa2841a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.429+0000 7fa283e00640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa284075540 0x7fa28419b4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa27400d770 tx=0x7fa27400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.429+0000 7fa280c00640 1 -- 172.21.15.103:0/1493778669 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274017070 con 0x7fa284075540 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.429+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2841a1890 con 0x7fa284075540 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.429+0000 7fa280c00640 1 -- 172.21.15.103:0/1493778669 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2740054d0 con 0x7fa284075540 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.429+0000 7fa280c00640 1 -- 172.21.15.103:0/1493778669 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274004c30 con 0x7fa284075540 2024-09-11T00:21:32.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.429+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2841a7f50 con 0x7fa284075540 2024-09-11T00:21:32.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.430+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa284075920 con 0x7fa284075540 2024-09-11T00:21:32.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.432+0000 7fa280c00640 1 -- 172.21.15.103:0/1493778669 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 16) ==== 153251+0+0 (secure 0 0 0) 0x7fa274049aa0 con 0x7fa284075540 2024-09-11T00:21:32.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.433+0000 7fa280c00640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa2680b52e0 0x7fa2680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:32.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.434+0000 7fa280c00640 1 -- 172.21.15.103:0/1493778669 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa2740e5990 con 0x7fa284075540 2024-09-11T00:21:32.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.434+0000 7fa283400640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa2680b52e0 0x7fa2680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:32.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.435+0000 7fa283400640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa2680b52e0 0x7fa2680b77a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa270005ab0 tx=0x7fa270005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:32.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.435+0000 7fa280c00640 1 -- 172.21.15.103:0/1493778669 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa2740abe40 con 0x7fa284075540 2024-09-11T00:21:32.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:32 smithi183 ceph-mon[35934]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:32.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.646+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa284100440 con 0x7fa2680b52e0 2024-09-11T00:21:32.648 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:32 smithi103 ceph-mon[30578]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:32.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.672+0000 7fa280c00640 1 -- 172.21.15.103:0/1493778669 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa284100440 con 0x7fa2680b52e0 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa2680b52e0 msgr2=0x7fa2680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa2680b52e0 0x7fa2680b77a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa270005ab0 tx=0x7fa270005a20 comp rx=0 tx=0).stop 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa284075540 msgr2=0x7fa28419b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa284075540 0x7fa28419b4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa27400d770 tx=0x7fa27400dc30 comp rx=0 tx=0).stop 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 shutdown_connections 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa2680b52e0 0x7fa2680b77a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa28410f3e0 0x7fa2841a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa284075f40 0x7fa28419ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.677 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 --2- 172.21.15.103:0/1493778669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa284075540 0x7fa28419b4f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:32.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 >> 172.21.15.103:0/1493778669 conn(0x7fa2840fe830 msgr2=0x7fa2841115f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:32.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 shutdown_connections 2024-09-11T00:21:32.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:32.675+0000 7fa28b200640 1 -- 172.21.15.103:0/1493778669 wait complete. 2024-09-11T00:21:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:32 smithi117 ceph-mon[35525]: from='client.24821 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:33.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:33 smithi183 ceph-mon[35934]: pgmap v350: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 383 B/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:33.602 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:33 smithi103 ceph-mon[30578]: pgmap v350: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 383 B/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:33.640 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T00:21:33.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:33 smithi117 ceph-mon[35525]: pgmap v350: 129 pgs: 129 active+clean; 584 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 383 B/s wr, 9 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:34.012 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:34.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:34 smithi103 ceph-mon[30578]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:34 smithi103 ceph-mon[30578]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T00:21:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:34 smithi103 ceph-mon[30578]: Cluster is now healthy 2024-09-11T00:21:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:34 smithi103 ceph-mon[30578]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-09-11T00:21:34.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:34 smithi183 ceph-mon[35934]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:34.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:34 smithi183 ceph-mon[35934]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T00:21:34.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:34 smithi183 ceph-mon[35934]: Cluster is now healthy 2024-09-11T00:21:34.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:34 smithi183 ceph-mon[35934]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-09-11T00:21:34.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 -- 172.21.15.103:0/896193804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7654104df0 msgr2=0x7f76541051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 --2- 172.21.15.103:0/896193804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7654104df0 0x7f76541051d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7644009900 tx=0x7f764402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 -- 172.21.15.103:0/896193804 shutdown_connections 2024-09-11T00:21:34.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 --2- 172.21.15.103:0/896193804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f765410b180 0x7f7654111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 --2- 172.21.15.103:0/896193804 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410a800 0x7f765410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.666 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 --2- 172.21.15.103:0/896193804 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7654104df0 0x7f76541051d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 -- 172.21.15.103:0/896193804 >> 172.21.15.103:0/896193804 conn(0x7f7654100840 msgr2=0x7f7654102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 -- 172.21.15.103:0/896193804 shutdown_connections 2024-09-11T00:21:34.667 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.664+0000 7f7653e00640 1 -- 172.21.15.103:0/896193804 wait complete. 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 Processor -- start 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 -- start start 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7654104df0 0x7f765419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f765410a800 0x7f765419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410b180 0x7f76541a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7654114e90 con 0x7f765410a800 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7654114d10 con 0x7f765410b180 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.665+0000 7f7653e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7654115010 con 0x7f7654104df0 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7652000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f765410a800 0x7f765419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410b180 0x7f76541a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7652a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7654104df0 0x7f765419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410b180 0x7f76541a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33024/0 (socket says 172.21.15.103:33024) 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 -- 172.21.15.103:0/2668432796 learned_addr learned my addr 172.21.15.103:0/2668432796 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 -- 172.21.15.103:0/2668432796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7654104df0 msgr2=0x7f765419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7654104df0 0x7f765419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 -- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f765410a800 msgr2=0x7f765419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f765410a800 0x7f765419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 -- 172.21.15.103:0/2668432796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76541a12b0 con 0x7f765410b180 2024-09-11T00:21:34.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.666+0000 7f7653400640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410b180 0x7f76541a0ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f764c00eb10 tx=0x7f764c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.667+0000 7f764b400640 1 -- 172.21.15.103:0/2668432796 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c004b20 con 0x7f765410b180 2024-09-11T00:21:34.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.667+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76541a15a0 con 0x7f765410b180 2024-09-11T00:21:34.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.667+0000 7f764b400640 1 -- 172.21.15.103:0/2668432796 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f764c004cc0 con 0x7f765410b180 2024-09-11T00:21:34.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.667+0000 7f764b400640 1 -- 172.21.15.103:0/2668432796 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c010690 con 0x7f765410b180 2024-09-11T00:21:34.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.667+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76541a7e30 con 0x7f765410b180 2024-09-11T00:21:34.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.668+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76541051d0 con 0x7f765410b180 2024-09-11T00:21:34.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.670+0000 7f764b400640 1 -- 172.21.15.103:0/2668432796 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f764c005020 con 0x7f765410b180 2024-09-11T00:21:34.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.671+0000 7f764b400640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76380b5490 0x7f76380b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:34.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.672+0000 7f764b400640 1 -- 172.21.15.103:0/2668432796 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f764c0e6610 con 0x7f765410b180 2024-09-11T00:21:34.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.672+0000 7f7652a00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76380b5490 0x7f76380b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:34.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.672+0000 7f7652a00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76380b5490 0x7f76380b7950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7644005ab0 tx=0x7f7644005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:34.675 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.674+0000 7f764b400640 1 -- 172.21.15.103:0/2668432796 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f764c0acfb0 con 0x7f765410b180 2024-09-11T00:21:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:34 smithi117 ceph-mon[35525]: from='client.24833 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:34 smithi117 ceph-mon[35525]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-11T00:21:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:34 smithi117 ceph-mon[35525]: Cluster is now healthy 2024-09-11T00:21:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:34 smithi117 ceph-mon[35525]: mgrmap e17: x(active, since 9m), standbys: z, y 2024-09-11T00:21:34.889 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.888+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7654102580 con 0x7f76380b5490 2024-09-11T00:21:34.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.895+0000 7f764b400640 1 -- 172.21.15.103:0/2668432796 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7654102580 con 0x7f76380b5490 2024-09-11T00:21:34.897 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_0/9b16fb4a-801c-48be-86db-d3862aa52b3b 2024-09-11T00:21:34.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76380b5490 msgr2=0x7f76380b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76380b5490 0x7f76380b7950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7644005ab0 tx=0x7f7644005a20 comp rx=0 tx=0).stop 2024-09-11T00:21:34.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410b180 msgr2=0x7f76541a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:34.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410b180 0x7f76541a0ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f764c00eb10 tx=0x7f764c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 shutdown_connections 2024-09-11T00:21:34.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f76380b5490 0x7f76380b7950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f765410b180 0x7f76541a0ae0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f765410a800 0x7f765419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 --2- 172.21.15.103:0/2668432796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7654104df0 0x7f765419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 >> 172.21.15.103:0/2668432796 conn(0x7f7654100840 msgr2=0x7f7654101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 shutdown_connections 2024-09-11T00:21:34.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:34.897+0000 7f7653e00640 1 -- 172.21.15.103:0/2668432796 wait complete. 2024-09-11T00:21:35.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:35 smithi183 ceph-mon[35934]: pgmap v351: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.5 KiB/s wr, 13 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:35.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:35 smithi103 ceph-mon[30578]: pgmap v351: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.5 KiB/s wr, 13 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:35 smithi117 ceph-mon[35525]: pgmap v351: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.5 KiB/s wr, 13 op/s; 13 B/s, 0 objects/s recovering 2024-09-11T00:21:35.727 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T00:21:36.101 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:36.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:36 smithi183 ceph-mon[35934]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:36.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:36 smithi103 ceph-mon[30578]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:36 smithi117 ceph-mon[35525]: from='client.24839 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 -- 172.21.15.103:0/1394955658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6970104df0 msgr2=0x7f69701051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 --2- 172.21.15.103:0/1394955658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6970104df0 0x7f69701051d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6964009900 tx=0x7f696402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 -- 172.21.15.103:0/1394955658 shutdown_connections 2024-09-11T00:21:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 --2- 172.21.15.103:0/1394955658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f697010b180 0x7f6970111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 --2- 172.21.15.103:0/1394955658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f697010a800 0x7f697010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 --2- 172.21.15.103:0/1394955658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6970104df0 0x7f69701051d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 -- 172.21.15.103:0/1394955658 >> 172.21.15.103:0/1394955658 conn(0x7f6970100840 msgr2=0x7f6970102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 -- 172.21.15.103:0/1394955658 shutdown_connections 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.693+0000 7f6976200640 1 -- 172.21.15.103:0/1394955658 wait complete. 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.694+0000 7f6976200640 1 Processor -- start 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.694+0000 7f6976200640 1 -- start start 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.694+0000 7f6976200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6970104df0 0x7f697019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.694+0000 7f6976200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f697010a800 0x7f697019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6976200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f697010b180 0x7f69701a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6976200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6970114e40 con 0x7f697010b180 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6976200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6970114cc0 con 0x7f6970104df0 2024-09-11T00:21:36.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6976200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6970114fc0 con 0x7f697010a800 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f697010b180 0x7f69701a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f697010b180 0x7f69701a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36862/0 (socket says 172.21.15.103:36862) 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 -- 172.21.15.103:0/2360548467 learned_addr learned my addr 172.21.15.103:0/2360548467 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6974e00640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6970104df0 0x7f697019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f696fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f697010a800 0x7f697019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 -- 172.21.15.103:0/2360548467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f697010a800 msgr2=0x7f697019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f697010a800 0x7f697019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 -- 172.21.15.103:0/2360548467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6970104df0 msgr2=0x7f697019b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6970104df0 0x7f697019b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.695+0000 7f6975800640 1 -- 172.21.15.103:0/2360548467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69701a12d0 con 0x7f697010b180 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.696+0000 7f6975800640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f697010b180 0x7f69701a0b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f696800eb10 tx=0x7f696800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:36.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.696+0000 7f696d600640 1 -- 172.21.15.103:0/2360548467 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968004b20 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.696+0000 7f696d600640 1 -- 172.21.15.103:0/2360548467 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6968004cc0 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.696+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69701a15c0 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.696+0000 7f696d600640 1 -- 172.21.15.103:0/2360548467 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968010670 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.696+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69701a7c80 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.697+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69701051d0 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.699+0000 7f696d600640 1 -- 172.21.15.103:0/2360548467 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f6968005020 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.700+0000 7f696d600640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69580b5490 0x7f69580b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.701+0000 7f696d600640 1 -- 172.21.15.103:0/2360548467 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f69680e6690 con 0x7f697010b180 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.701+0000 7f6974e00640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69580b5490 0x7f69580b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.702+0000 7f6974e00640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69580b5490 0x7f69580b7950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6964031440 tx=0x7f6964005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:36.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.704+0000 7f696d600640 1 -- 172.21.15.103:0/2360548467 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f69680ad030 con 0x7f697010b180 2024-09-11T00:21:36.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.913+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6970063590 con 0x7f69580b5490 2024-09-11T00:21:36.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.946+0000 7f696d600640 1 -- 172.21.15.103:0/2360548467 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6970063590 con 0x7f69580b5490 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.948+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69580b5490 msgr2=0x7f69580b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.948+0000 7f6976200640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69580b5490 0x7f69580b7950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6964031440 tx=0x7f6964005b50 comp rx=0 tx=0).stop 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.948+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f697010b180 msgr2=0x7f69701a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.948+0000 7f6976200640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f697010b180 0x7f69701a0b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f696800eb10 tx=0x7f696800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 shutdown_connections 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69580b5490 0x7f69580b7950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f697010b180 0x7f69701a0b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f697010a800 0x7f697019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 --2- 172.21.15.103:0/2360548467 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6970104df0 0x7f697019b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 >> 172.21.15.103:0/2360548467 conn(0x7f6970100840 msgr2=0x7f6970106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:36.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 shutdown_connections 2024-09-11T00:21:36.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:36.949+0000 7f6976200640 1 -- 172.21.15.103:0/2360548467 wait complete. 2024-09-11T00:21:37.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:37 smithi183 ceph-mon[35934]: pgmap v352: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s; 12 B/s, 0 objects/s recovering 2024-09-11T00:21:37.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:37 smithi103 ceph-mon[30578]: pgmap v352: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s; 12 B/s, 0 objects/s recovering 2024-09-11T00:21:37.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:37 smithi117 ceph-mon[35525]: pgmap v352: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.1 KiB/s wr, 12 op/s; 12 B/s, 0 objects/s recovering 2024-09-11T00:21:37.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T00:21:38.173 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:38.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:38 smithi183 ceph-mon[35934]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:38.540 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:38 smithi103 ceph-mon[30578]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:38 smithi117 ceph-mon[35525]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:38.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.798+0000 7f8de2800640 1 -- 172.21.15.103:0/255251828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc104df0 msgr2=0x7f8ddc1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:38.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.798+0000 7f8de2800640 1 --2- 172.21.15.103:0/255251828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc104df0 0x7f8ddc1051d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009ca0 tx=0x7f8dd002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:38.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.798+0000 7f8de2800640 1 -- 172.21.15.103:0/255251828 shutdown_connections 2024-09-11T00:21:38.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.798+0000 7f8de2800640 1 --2- 172.21.15.103:0/255251828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc10b180 0x7f8ddc111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:38.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.798+0000 7f8de2800640 1 --2- 172.21.15.103:0/255251828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc10a800 0x7f8ddc10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:38.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.798+0000 7f8de2800640 1 --2- 172.21.15.103:0/255251828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc104df0 0x7f8ddc1051d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.798+0000 7f8de2800640 1 -- 172.21.15.103:0/255251828 >> 172.21.15.103:0/255251828 conn(0x7f8ddc100840 msgr2=0x7f8ddc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.799+0000 7f8de2800640 1 -- 172.21.15.103:0/255251828 shutdown_connections 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.799+0000 7f8de2800640 1 -- 172.21.15.103:0/255251828 wait complete. 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 Processor -- start 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 -- start start 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc104df0 0x7f8ddc19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc10a800 0x7f8ddc19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc10b180 0x7f8ddc1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:38.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc114e90 con 0x7f8ddc10a800 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc114d10 con 0x7f8ddc10b180 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de2800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8ddc115010 con 0x7f8ddc104df0 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc10a800 0x7f8ddc19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc10a800 0x7f8ddc19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36868/0 (socket says 172.21.15.103:36868) 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de0a00640 1 -- 172.21.15.103:0/2535658460 learned_addr learned my addr 172.21.15.103:0/2535658460 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.800+0000 7f8de1400640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc104df0 0x7f8ddc19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de1e00640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc10b180 0x7f8ddc1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:38.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de0a00640 1 -- 172.21.15.103:0/2535658460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc104df0 msgr2=0x7f8ddc19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de0a00640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc104df0 0x7f8ddc19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de0a00640 1 -- 172.21.15.103:0/2535658460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc10b180 msgr2=0x7f8ddc1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de0a00640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc10b180 0x7f8ddc1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de0a00640 1 -- 172.21.15.103:0/2535658460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ddc1a12e0 con 0x7f8ddc10a800 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de0a00640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc10a800 0x7f8ddc19b770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e9f0 tx=0x7f8dc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8dda000640 1 -- 172.21.15.103:0/2535658460 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800cd80 con 0x7f8ddc10a800 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8dda000640 1 -- 172.21.15.103:0/2535658460 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dc80054d0 con 0x7f8ddc10a800 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ddc1a15d0 con 0x7f8ddc10a800 2024-09-11T00:21:38.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8dda000640 1 -- 172.21.15.103:0/2535658460 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8010690 con 0x7f8ddc10a800 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.801+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ddc1a7c90 con 0x7f8ddc10a800 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.802+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ddc1051d0 con 0x7f8ddc10a800 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.805+0000 7f8dda000640 1 -- 172.21.15.103:0/2535658460 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f8dc8004e90 con 0x7f8ddc10a800 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.806+0000 7f8dda000640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dc40b5490 0x7f8dc40b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.806+0000 7f8dda000640 1 -- 172.21.15.103:0/2535658460 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f8dc80e65b0 con 0x7f8ddc10a800 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.806+0000 7f8de1400640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dc40b5490 0x7f8dc40b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.807+0000 7f8de1400640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dc40b5490 0x7f8dc40b7950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0006010 tx=0x7f8dd0009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:38.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:38.810+0000 7f8dda000640 1 -- 172.21.15.103:0/2535658460 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8dc80acf50 con 0x7f8ddc10a800 2024-09-11T00:21:39.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.021+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8ddc107dc0 con 0x7f8dc40b5490 2024-09-11T00:21:39.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.027+0000 7f8dda000640 1 -- 172.21.15.103:0/2535658460 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8ddc107dc0 con 0x7f8dc40b5490 2024-09-11T00:21:39.029 INFO:teuthology.orchestra.run.smithi103.stdout:/volumes/qa/sv_1/e9949dac-966e-4aba-acd3-2eaad7312f6e 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.029+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dc40b5490 msgr2=0x7f8dc40b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.029+0000 7f8de2800640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dc40b5490 0x7f8dc40b7950 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0006010 tx=0x7f8dd0009820 comp rx=0 tx=0).stop 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.029+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc10a800 msgr2=0x7f8ddc19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.029+0000 7f8de2800640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc10a800 0x7f8ddc19b770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e9f0 tx=0x7f8dc800edf0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 shutdown_connections 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dc40b5490 0x7f8dc40b7950 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc10b180 0x7f8ddc1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ddc10a800 0x7f8ddc19b770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 --2- 172.21.15.103:0/2535658460 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8ddc104df0 0x7f8ddc19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:39.032 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 >> 172.21.15.103:0/2535658460 conn(0x7f8ddc100840 msgr2=0x7f8ddc109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 shutdown_connections 2024-09-11T00:21:39.033 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:39.030+0000 7f8de2800640 1 -- 172.21.15.103:0/2535658460 wait complete. 2024-09-11T00:21:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:39 smithi183 ceph-mon[35934]: pgmap v353: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-11T00:21:39.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:39 smithi103 ceph-mon[30578]: pgmap v353: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-11T00:21:39.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:39 smithi117 ceph-mon[35525]: pgmap v353: 129 pgs: 129 active+clean; 605 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 9 op/s; 11 B/s, 0 objects/s recovering 2024-09-11T00:21:39.948 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:21:40.321 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:40.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:40 smithi183 ceph-mon[35934]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:40.626 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:40 smithi103 ceph-mon[30578]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:40 smithi117 ceph-mon[35525]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 -- 172.21.15.103:0/3437586718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 msgr2=0x7f526c111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 --2- 172.21.15.103:0/3437586718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c111b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5264009ca0 tx=0x7f526402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 -- 172.21.15.103:0/3437586718 shutdown_connections 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 --2- 172.21.15.103:0/3437586718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c111b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 --2- 172.21.15.103:0/3437586718 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526c10a9d0 0x7f526c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 --2- 172.21.15.103:0/3437586718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526c104fc0 0x7f526c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 -- 172.21.15.103:0/3437586718 >> 172.21.15.103:0/3437586718 conn(0x7f526c1009f0 msgr2=0x7f526c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:40.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 -- 172.21.15.103:0/3437586718 shutdown_connections 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.925+0000 7f5273800640 1 -- 172.21.15.103:0/3437586718 wait complete. 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 Processor -- start 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 -- start start 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526c104fc0 0x7f526c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526c10a9d0 0x7f526c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f526c114fd0 con 0x7f526c10b350 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f526c114e50 con 0x7f526c10a9d0 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.926+0000 7f5273800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f526c115150 con 0x7f526c104fc0 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5270e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526c104fc0 0x7f526c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:40.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c1a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36882/0 (socket says 172.21.15.103:36882) 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 -- 172.21.15.103:0/678815908 learned_addr learned my addr 172.21.15.103:0/678815908 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f526be00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526c10a9d0 0x7f526c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 -- 172.21.15.103:0/678815908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526c104fc0 msgr2=0x7f526c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526c104fc0 0x7f526c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 -- 172.21.15.103:0/678815908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526c10a9d0 msgr2=0x7f526c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526c10a9d0 0x7f526c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 -- 172.21.15.103:0/678815908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c1a1510 con 0x7f526c10b350 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f526be00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526c10a9d0 0x7f526c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.927+0000 7f5271800640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c1a0d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f526402f540 tx=0x7f52640316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.928+0000 7f5269600640 1 -- 172.21.15.103:0/678815908 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264003b70 con 0x7f526c10b350 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.928+0000 7f5269600640 1 -- 172.21.15.103:0/678815908 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5264003d10 con 0x7f526c10b350 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.928+0000 7f5269600640 1 -- 172.21.15.103:0/678815908 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526402f990 con 0x7f526c10b350 2024-09-11T00:21:40.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.928+0000 7f5273800640 1 -- 172.21.15.103:0/678815908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c1a17a0 con 0x7f526c10b350 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.928+0000 7f5273800640 1 -- 172.21.15.103:0/678815908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a7ec0 con 0x7f526c10b350 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.929+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c1053a0 con 0x7f526c10b350 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.931+0000 7f5269600640 1 -- 172.21.15.103:0/678815908 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f526402fb30 con 0x7f526c10b350 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.932+0000 7f5269600640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f52540b53f0 0x7f52540b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.932+0000 7f5269600640 1 -- 172.21.15.103:0/678815908 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f5264107210 con 0x7f526c10b350 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.932+0000 7f5270e00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f52540b53f0 0x7f52540b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.933+0000 7f5270e00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f52540b53f0 0x7f52540b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5260009bd0 tx=0x7f5260005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:40.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:40.936+0000 7f5269600640 1 -- 172.21.15.103:0/678815908 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52640cdbb0 con 0x7f526c10b350 2024-09-11T00:21:41.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.156+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f526c063d50 con 0x7f526c10b350 2024-09-11T00:21:41.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.157+0000 7f5269600640 1 -- 172.21.15.103:0/678815908 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f52640cd4b0 con 0x7f526c10b350 2024-09-11T00:21:41.160 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:41.160 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-11T00:21:28:210497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:21:27.247376+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:21:41.160 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.159+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f52540b53f0 msgr2=0x7f52540b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.159+0000 7f525ea00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f52540b53f0 0x7f52540b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5260009bd0 tx=0x7f5260005f90 comp rx=0 tx=0).stop 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.159+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 msgr2=0x7f526c1a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c1a0d40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f526402f540 tx=0x7f52640316f0 comp rx=0 tx=0).stop 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 shutdown_connections 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f52540b53f0 0x7f52540b78b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f526c10b350 0x7f526c1a0d40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f526c10a9d0 0x7f526c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 --2- 172.21.15.103:0/678815908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f526c104fc0 0x7f526c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:41.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 >> 172.21.15.103:0/678815908 conn(0x7f526c1009f0 msgr2=0x7f526c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:41.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 shutdown_connections 2024-09-11T00:21:41.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:41.160+0000 7f525ea00640 1 -- 172.21.15.103:0/678815908 wait complete. 2024-09-11T00:21:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:41 smithi183 ceph-mon[35934]: pgmap v354: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:21:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:41 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/678815908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:41.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:41 smithi103 ceph-mon[30578]: pgmap v354: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:21:41.630 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:41 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/678815908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:41 smithi117 ceph-mon[35525]: pgmap v354: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.2 KiB/s wr, 10 op/s; 9 B/s, 0 objects/s recovering 2024-09-11T00:21:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:41 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/678815908' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:21:42.187 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:21:42.559 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:42.698 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:42 smithi103 ceph-mon[30578]: pgmap v355: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-11T00:21:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:42 smithi183 ceph-mon[35934]: pgmap v355: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-11T00:21:42.866 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:42 smithi117 ceph-mon[35525]: pgmap v355: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-09-11T00:21:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1118029363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 msgr2=0x7fe2b0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:43.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1118029363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 0x7fe2b0108a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0009900 tx=0x7fe2a002a400 comp rx=0 tx=0).stop 2024-09-11T00:21:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1118029363 shutdown_connections 2024-09-11T00:21:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1118029363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0108fa0 0x7fe2b010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1118029363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 0x7fe2b0108a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1118029363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2b0102c10 0x7fe2b0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1118029363 >> 172.21.15.103:0/1118029363 conn(0x7fe2b00fe660 msgr2=0x7fe2b0100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1118029363 shutdown_connections 2024-09-11T00:21:43.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.187+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1118029363 wait complete. 2024-09-11T00:21:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.188+0000 7fe2b4a00640 1 Processor -- start 2024-09-11T00:21:43.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.188+0000 7fe2b4a00640 1 -- start start 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2b4a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0102c10 0x7fe2b019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2b4a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 0x7fe2b019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2b4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2b0108fa0 0x7fe2b01a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2b4a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0112c70 con 0x7fe2b0108fa0 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2b4a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0112af0 con 0x7fe2b0102c10 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2b4a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0112df0 con 0x7fe2b0108620 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0102c10 0x7fe2b019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0102c10 0x7fe2b019b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:47482/0 (socket says 172.21.15.103:47482) 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2afe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2b0108fa0 0x7fe2b01a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 -- 172.21.15.103:0/1544132939 learned_addr learned my addr 172.21.15.103:0/1544132939 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2aea00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 0x7fe2b019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 -- 172.21.15.103:0/1544132939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 msgr2=0x7fe2b019b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 0x7fe2b019b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 -- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2b0108fa0 msgr2=0x7fe2b01a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2b0108fa0 0x7fe2b01a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.189+0000 7fe2af400640 1 -- 172.21.15.103:0/1544132939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b01a1330 con 0x7fe2b0102c10 2024-09-11T00:21:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.190+0000 7fe2af400640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0102c10 0x7fe2b019b280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9f0 tx=0x7fe29800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:43.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.190+0000 7fe2a7e00640 1 -- 172.21.15.103:0/1544132939 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800cd80 con 0x7fe2b0102c10 2024-09-11T00:21:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.190+0000 7fe2a7e00640 1 -- 172.21.15.103:0/1544132939 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2980108e0 con 0x7fe2b0102c10 2024-09-11T00:21:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.190+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b01a1620 con 0x7fe2b0102c10 2024-09-11T00:21:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.190+0000 7fe2a7e00640 1 -- 172.21.15.103:0/1544132939 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298018980 con 0x7fe2b0102c10 2024-09-11T00:21:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.190+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b01a7ce0 con 0x7fe2b0102c10 2024-09-11T00:21:43.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.191+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b0102ff0 con 0x7fe2b0102c10 2024-09-11T00:21:43.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.193+0000 7fe2a7e00640 1 -- 172.21.15.103:0/1544132939 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fe298010430 con 0x7fe2b0102c10 2024-09-11T00:21:43.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.194+0000 7fe2a7e00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2940b5410 0x7fe2940b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.195+0000 7fe2a7e00640 1 -- 172.21.15.103:0/1544132939 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe2980e6470 con 0x7fe2b0102c10 2024-09-11T00:21:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.195+0000 7fe2aea00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2940b5410 0x7fe2940b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:43.197 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.195+0000 7fe2aea00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2940b5410 0x7fe2940b78d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005ab0 tx=0x7fe2a0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:43.200 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.198+0000 7fe2a7e00640 1 -- 172.21.15.103:0/1544132939 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe2980ac7c0 con 0x7fe2b0102c10 2024-09-11T00:21:43.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.384+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b002e5c0 con 0x7fe2b0102c10 2024-09-11T00:21:43.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.386+0000 7fe2a7e00640 1 -- 172.21.15.103:0/1544132939 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24772 (secure 0 0 0) 0x7fe2980b11f0 con 0x7fe2b0102c10 2024-09-11T00:21:43.388 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:43.388 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":87,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:21:28.211213+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:21:43.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2940b5410 msgr2=0x7fe2940b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:43.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2940b5410 0x7fe2940b78d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0005ab0 tx=0x7fe2a0005a20 comp rx=0 tx=0).stop 2024-09-11T00:21:43.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0102c10 msgr2=0x7fe2b019b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:43.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0102c10 0x7fe2b019b280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e9f0 tx=0x7fe29800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 shutdown_connections 2024-09-11T00:21:43.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2940b5410 0x7fe2940b78d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2b0108fa0 0x7fe2b01a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2b0108620 0x7fe2b019b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 --2- 172.21.15.103:0/1544132939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2b0102c10 0x7fe2b019b280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:43.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 >> 172.21.15.103:0/1544132939 conn(0x7fe2b00fe660 msgr2=0x7fe2b0105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:43.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 shutdown_connections 2024-09-11T00:21:43.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:43.388+0000 7fe2b4a00640 1 -- 172.21.15.103:0/1544132939 wait complete. 2024-09-11T00:21:43.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:43 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1544132939' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:43.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:43 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1544132939' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:43.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:43 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1544132939' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:44.453 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:21:44.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:21:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:44 smithi103 ceph-mon[30578]: pgmap v356: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-11T00:21:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:44 smithi183 ceph-mon[35934]: pgmap v356: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-11T00:21:44.844 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:44 smithi117 ceph-mon[35525]: pgmap v356: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-09-11T00:21:45.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 -- 172.21.15.103:0/3549249106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 msgr2=0x7f88d8111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 --2- 172.21.15.103:0/3549249106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d8111980 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009ca0 tx=0x7f88d002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 -- 172.21.15.103:0/3549249106 shutdown_connections 2024-09-11T00:21:45.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 --2- 172.21.15.103:0/3549249106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d8111980 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 --2- 172.21.15.103:0/3549249106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88d810a800 0x7f88d810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 --2- 172.21.15.103:0/3549249106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88d8104df0 0x7f88d81051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 -- 172.21.15.103:0/3549249106 >> 172.21.15.103:0/3549249106 conn(0x7f88d8100840 msgr2=0x7f88d8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 -- 172.21.15.103:0/3549249106 shutdown_connections 2024-09-11T00:21:45.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.517+0000 7f88d7200640 1 -- 172.21.15.103:0/3549249106 wait complete. 2024-09-11T00:21:45.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.518+0000 7f88d7200640 1 Processor -- start 2024-09-11T00:21:45.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.518+0000 7f88d7200640 1 -- start start 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88d8104df0 0x7f88d819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88d810a800 0x7f88d819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88d8114e50 con 0x7f88d810b180 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f88d8114cd0 con 0x7f88d8104df0 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88d8114fd0 con 0x7f88d810a800 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88d8104df0 0x7f88d819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d81a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35004/0 (socket says 172.21.15.103:35004) 2024-09-11T00:21:45.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d6800640 1 -- 172.21.15.103:0/2673018127 learned_addr learned my addr 172.21.15.103:0/2673018127 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.519+0000 7f88d5400640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88d810a800 0x7f88d819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d5400640 1 -- 172.21.15.103:0/2673018127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88d8104df0 msgr2=0x7f88d819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d5400640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88d8104df0 0x7f88d819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d5400640 1 -- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 msgr2=0x7f88d81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d5400640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d5400640 1 -- 172.21.15.103:0/2673018127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d81a1350 con 0x7f88d810a800 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d6800640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d5400640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88d810a800 0x7f88d819b7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e9f0 tx=0x7f88c400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88cea00640 1 -- 172.21.15.103:0/2673018127 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c400ccc0 con 0x7f88d810a800 2024-09-11T00:21:45.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d81a1640 con 0x7f88d810a800 2024-09-11T00:21:45.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88cea00640 1 -- 172.21.15.103:0/2673018127 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c40108e0 con 0x7f88d810a800 2024-09-11T00:21:45.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.520+0000 7f88cea00640 1 -- 172.21.15.103:0/2673018127 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4018880 con 0x7f88d810a800 2024-09-11T00:21:45.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.521+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d81a7ed0 con 0x7f88d810a800 2024-09-11T00:21:45.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.522+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d81051d0 con 0x7f88d810a800 2024-09-11T00:21:45.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.524+0000 7f88cea00640 1 -- 172.21.15.103:0/2673018127 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f88c4010430 con 0x7f88d810a800 2024-09-11T00:21:45.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.525+0000 7f88cea00640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88bc0b5490 0x7f88bc0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:45.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.525+0000 7f88cea00640 1 -- 172.21.15.103:0/2673018127 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f88c40e6830 con 0x7f88d810a800 2024-09-11T00:21:45.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.525+0000 7f88d5e00640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88bc0b5490 0x7f88bc0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:45.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.526+0000 7f88d5e00640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88bc0b5490 0x7f88bc0b7950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f88c8005fd0 tx=0x7f88c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:45.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.527+0000 7f88cea00640 1 -- 172.21.15.103:0/2673018127 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f88c40acb80 con 0x7f88d810a800 2024-09-11T00:21:45.711 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.709+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f88d8063590 con 0x7f88bc0b5490 2024-09-11T00:21:45.718 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.717+0000 7f88cea00640 1 -- 172.21.15.103:0/2673018127 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340349 (secure 0 0 0) 0x7f88d8063590 con 0x7f88bc0b5490 2024-09-11T00:21:45.723 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:45.725 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-11T00:21:45.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88bc0b5490 msgr2=0x7f88bc0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88bc0b5490 0x7f88bc0b7950 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f88c8005fd0 tx=0x7f88c800b390 comp rx=0 tx=0).stop 2024-09-11T00:21:45.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88d810a800 msgr2=0x7f88d819b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88d810a800 0x7f88d819b7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f88c400e9f0 tx=0x7f88c400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 shutdown_connections 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88bc0b5490 0x7f88bc0b7950 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88d810b180 0x7f88d81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88d810a800 0x7f88d819b7e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 --2- 172.21.15.103:0/2673018127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f88d8104df0 0x7f88d819b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 >> 172.21.15.103:0/2673018127 conn(0x7f88d8100840 msgr2=0x7f88d8100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.721+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 shutdown_connections 2024-09-11T00:21:45.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:45.722+0000 7f88d7200640 1 -- 172.21.15.103:0/2673018127 wait complete. 2024-09-11T00:21:46.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:46 smithi103 ceph-mon[30578]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:21:46.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:46 smithi103 ceph-mon[30578]: pgmap v357: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-11T00:21:46.777 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-11T00:21:44.377895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647007,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":212,"num_read_kb":190,"num_write":286,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375416,"kb_used_data":13676,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566408,"statfs":{"total":1151940427776,"available":1151556001792,"internally_reserved":0,"allocated":14004224,"data_stored":7871735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49187,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":90,"num_write":32,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003335"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225954+0000","last_change":"2024-09-11T00:20:24.027578+0000","last_active":"2024-09-11T00:21:28.225954+0000","last_peered":"2024-09-11T00:21:28.225954+0000","last_clean":"2024-09-11T00:21:28.225954+0000","last_became_active":"2024-09-11T00:20:24.027174+0000","last_became_peered":"2024-09-11T00:20:24.027174+0000","last_unstale":"2024-09-11T00:21:28.225954+0000","last_undegraded":"2024-09-11T00:21:28.225954+0000","last_fullsized":"2024-09-11T00:21:28.225954+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:15.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220553+0000","last_change":"2024-09-11T00:20:27.064047+0000","last_active":"2024-09-11T00:21:28.220553+0000","last_peered":"2024-09-11T00:21:28.220553+0000","last_clean":"2024-09-11T00:21:28.220553+0000","last_became_active":"2024-09-11T00:20:27.063756+0000","last_became_peered":"2024-09-11T00:20:27.063756+0000","last_unstale":"2024-09-11T00:21:28.220553+0000","last_undegraded":"2024-09-11T00:21:28.220553+0000","last_fullsized":"2024-09-11T00:21:28.220553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.695888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228288+0000","last_change":"2024-09-11T00:20:37.449428+0000","last_active":"2024-09-11T00:21:28.228288+0000","last_peered":"2024-09-11T00:21:28.228288+0000","last_clean":"2024-09-11T00:21:28.228288+0000","last_became_active":"2024-09-11T00:20:37.449015+0000","last_became_peered":"2024-09-11T00:20:37.449015+0000","last_unstale":"2024-09-11T00:21:28.228288+0000","last_undegraded":"2024-09-11T00:21:28.228288+0000","last_fullsized":"2024-09-11T00:21:28.228288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:42.983990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220947+0000","last_change":"2024-09-11T00:20:24.035966+0000","last_active":"2024-09-11T00:21:28.220947+0000","last_peered":"2024-09-11T00:21:28.220947+0000","last_clean":"2024-09-11T00:21:28.220947+0000","last_became_active":"2024-09-11T00:20:24.035663+0000","last_became_peered":"2024-09-11T00:20:24.035663+0000","last_unstale":"2024-09-11T00:21:28.220947+0000","last_undegraded":"2024-09-11T00:21:28.220947+0000","last_fullsized":"2024-09-11T00:21:28.220947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:22.529392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228283+0000","last_change":"2024-09-11T00:20:27.066470+0000","last_active":"2024-09-11T00:21:28.228283+0000","last_peered":"2024-09-11T00:21:28.228283+0000","last_clean":"2024-09-11T00:21:28.228283+0000","last_became_active":"2024-09-11T00:20:27.065424+0000","last_became_peered":"2024-09-11T00:20:27.065424+0000","last_unstale":"2024-09-11T00:21:28.228283+0000","last_undegraded":"2024-09-11T00:21:28.228283+0000","last_fullsized":"2024-09-11T00:21:28.228283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:39.967332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220052+0000","last_change":"2024-09-11T00:20:37.453105+0000","last_active":"2024-09-11T00:21:28.220052+0000","last_peered":"2024-09-11T00:21:28.220052+0000","last_clean":"2024-09-11T00:21:28.220052+0000","last_became_active":"2024-09-11T00:20:37.452319+0000","last_became_peered":"2024-09-11T00:20:37.452319+0000","last_unstale":"2024-09-11T00:21:28.220052+0000","last_undegraded":"2024-09-11T00:21:28.220052+0000","last_fullsized":"2024-09-11T00:21:28.220052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:05.873046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224025+0000","last_change":"2024-09-11T00:20:24.022277+0000","last_active":"2024-09-11T00:21:28.224025+0000","last_peered":"2024-09-11T00:21:28.224025+0000","last_clean":"2024-09-11T00:21:28.224025+0000","last_became_active":"2024-09-11T00:20:24.021893+0000","last_became_peered":"2024-09-11T00:20:24.021893+0000","last_unstale":"2024-09-11T00:21:28.224025+0000","last_undegraded":"2024-09-11T00:21:28.224025+0000","last_fullsized":"2024-09-11T00:21:28.224025+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.994032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244372+0000","last_change":"2024-09-11T00:20:27.063368+0000","last_active":"2024-09-11T00:21:28.244372+0000","last_peered":"2024-09-11T00:21:28.244372+0000","last_clean":"2024-09-11T00:21:28.244372+0000","last_became_active":"2024-09-11T00:20:27.063051+0000","last_became_peered":"2024-09-11T00:20:27.063051+0000","last_unstale":"2024-09-11T00:21:28.244372+0000","last_undegraded":"2024-09-11T00:21:28.244372+0000","last_fullsized":"2024-09-11T00:21:28.244372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:09.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229139+0000","last_change":"2024-09-11T00:21:28.229073+0000","last_active":"2024-09-11T00:21:28.229139+0000","last_peered":"2024-09-11T00:21:28.229139+0000","last_clean":"2024-09-11T00:21:28.229139+0000","last_became_active":"2024-09-11T00:21:28.228775+0000","last_became_peered":"2024-09-11T00:21:28.228775+0000","last_unstale":"2024-09-11T00:21:28.229139+0000","last_undegraded":"2024-09-11T00:21:28.229139+0000","last_fullsized":"2024-09-11T00:21:28.229139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.886727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223773+0000","last_change":"2024-09-11T00:20:24.022562+0000","last_active":"2024-09-11T00:21:28.223773+0000","last_peered":"2024-09-11T00:21:28.223773+0000","last_clean":"2024-09-11T00:21:28.223773+0000","last_became_active":"2024-09-11T00:20:24.022207+0000","last_became_peered":"2024-09-11T00:20:24.022207+0000","last_unstale":"2024-09-11T00:21:28.223773+0000","last_undegraded":"2024-09-11T00:21:28.223773+0000","last_fullsized":"2024-09-11T00:21:28.223773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:01.071034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229830+0000","last_change":"2024-09-11T00:20:27.062572+0000","last_active":"2024-09-11T00:21:28.229830+0000","last_peered":"2024-09-11T00:21:28.229830+0000","last_clean":"2024-09-11T00:21:28.229830+0000","last_became_active":"2024-09-11T00:20:27.062288+0000","last_became_peered":"2024-09-11T00:20:27.062288+0000","last_unstale":"2024-09-11T00:21:28.229830+0000","last_undegraded":"2024-09-11T00:21:28.229830+0000","last_fullsized":"2024-09-11T00:21:28.229830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:37.391822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220548+0000","last_change":"2024-09-11T00:20:37.444028+0000","last_active":"2024-09-11T00:21:28.220548+0000","last_peered":"2024-09-11T00:21:28.220548+0000","last_clean":"2024-09-11T00:21:28.220548+0000","last_became_active":"2024-09-11T00:20:37.443344+0000","last_became_peered":"2024-09-11T00:20:37.443344+0000","last_unstale":"2024-09-11T00:21:28.220548+0000","last_undegraded":"2024-09-11T00:21:28.220548+0000","last_fullsized":"2024-09-11T00:21:28.220548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.347054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244654+0000","last_change":"2024-09-11T00:20:27.060317+0000","last_active":"2024-09-11T00:21:28.244654+0000","last_peered":"2024-09-11T00:21:28.244654+0000","last_clean":"2024-09-11T00:21:28.244654+0000","last_became_active":"2024-09-11T00:20:27.059983+0000","last_became_peered":"2024-09-11T00:20:27.059983+0000","last_unstale":"2024-09-11T00:21:28.244654+0000","last_undegraded":"2024-09-11T00:21:28.244654+0000","last_fullsized":"2024-09-11T00:21:28.244654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:28.405553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223622+0000","last_change":"2024-09-11T00:20:24.025555+0000","last_active":"2024-09-11T00:21:28.223622+0000","last_peered":"2024-09-11T00:21:28.223622+0000","last_clean":"2024-09-11T00:21:28.223622+0000","last_became_active":"2024-09-11T00:20:24.025171+0000","last_became_peered":"2024-09-11T00:20:24.025171+0000","last_unstale":"2024-09-11T00:21:28.223622+0000","last_undegraded":"2024-09-11T00:21:28.223622+0000","last_fullsized":"2024-09-11T00:21:28.223622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:22.708985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220409+0000","last_change":"2024-09-11T00:20:37.442057+0000","last_active":"2024-09-11T00:21:28.220409+0000","last_peered":"2024-09-11T00:21:28.220409+0000","last_clean":"2024-09-11T00:21:28.220409+0000","last_became_active":"2024-09-11T00:20:37.441555+0000","last_became_peered":"2024-09-11T00:20:37.441555+0000","last_unstale":"2024-09-11T00:21:28.220409+0000","last_undegraded":"2024-09-11T00:21:28.220409+0000","last_fullsized":"2024-09-11T00:21:28.220409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:08.155763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220312+0000","last_change":"2024-09-11T00:20:27.071533+0000","last_active":"2024-09-11T00:21:28.220312+0000","last_peered":"2024-09-11T00:21:28.220312+0000","last_clean":"2024-09-11T00:21:28.220312+0000","last_became_active":"2024-09-11T00:20:27.068771+0000","last_became_peered":"2024-09-11T00:20:27.068771+0000","last_unstale":"2024-09-11T00:21:28.220312+0000","last_undegraded":"2024-09-11T00:21:28.220312+0000","last_fullsized":"2024-09-11T00:21:28.220312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:05.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227708+0000","last_change":"2024-09-11T00:20:24.023282+0000","last_active":"2024-09-11T00:21:28.227708+0000","last_peered":"2024-09-11T00:21:28.227708+0000","last_clean":"2024-09-11T00:21:28.227708+0000","last_became_active":"2024-09-11T00:20:24.022914+0000","last_became_peered":"2024-09-11T00:20:24.022914+0000","last_unstale":"2024-09-11T00:21:28.227708+0000","last_undegraded":"2024-09-11T00:21:28.227708+0000","last_fullsized":"2024-09-11T00:21:28.227708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:45.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220653+0000","last_change":"2024-09-11T00:20:37.457338+0000","last_active":"2024-09-11T00:21:28.220653+0000","last_peered":"2024-09-11T00:21:28.220653+0000","last_clean":"2024-09-11T00:21:28.220653+0000","last_became_active":"2024-09-11T00:20:37.456993+0000","last_became_peered":"2024-09-11T00:20:37.456993+0000","last_unstale":"2024-09-11T00:21:28.220653+0000","last_undegraded":"2024-09-11T00:21:28.220653+0000","last_fullsized":"2024-09-11T00:21:28.220653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:57.284412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225994+0000","last_change":"2024-09-11T00:20:27.062752+0000","last_active":"2024-09-11T00:21:28.225994+0000","last_peered":"2024-09-11T00:21:28.225994+0000","last_clean":"2024-09-11T00:21:28.225994+0000","last_became_active":"2024-09-11T00:20:27.061569+0000","last_became_peered":"2024-09-11T00:20:27.061569+0000","last_unstale":"2024-09-11T00:21:28.225994+0000","last_undegraded":"2024-09-11T00:21:28.225994+0000","last_fullsized":"2024-09-11T00:21:28.225994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:30.904423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221935+0000","last_change":"2024-09-11T00:20:24.023197+0000","last_active":"2024-09-11T00:21:28.221935+0000","last_peered":"2024-09-11T00:21:28.221935+0000","last_clean":"2024-09-11T00:21:28.221935+0000","last_became_active":"2024-09-11T00:20:24.022867+0000","last_became_peered":"2024-09-11T00:20:24.022867+0000","last_unstale":"2024-09-11T00:21:28.221935+0000","last_undegraded":"2024-09-11T00:21:28.221935+0000","last_fullsized":"2024-09-11T00:21:28.221935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:37.949437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225776+0000","last_change":"2024-09-11T00:20:37.451731+0000","last_active":"2024-09-11T00:21:28.225776+0000","last_peered":"2024-09-11T00:21:28.225776+0000","last_clean":"2024-09-11T00:21:28.225776+0000","last_became_active":"2024-09-11T00:20:37.447710+0000","last_became_peered":"2024-09-11T00:20:37.447710+0000","last_unstale":"2024-09-11T00:21:28.225776+0000","last_undegraded":"2024-09-11T00:21:28.225776+0000","last_fullsized":"2024-09-11T00:21:28.225776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:11:38.471216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223166+0000","last_change":"2024-09-11T00:20:27.066021+0000","last_active":"2024-09-11T00:21:28.223166+0000","last_peered":"2024-09-11T00:21:28.223166+0000","last_clean":"2024-09-11T00:21:28.223166+0000","last_became_active":"2024-09-11T00:20:27.065652+0000","last_became_peered":"2024-09-11T00:20:27.065652+0000","last_unstale":"2024-09-11T00:21:28.223166+0000","last_undegraded":"2024-09-11T00:21:28.223166+0000","last_fullsized":"2024-09-11T00:21:28.223166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:01.933031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243482+0000","last_change":"2024-09-11T00:20:24.035619+0000","last_active":"2024-09-11T00:21:28.243482+0000","last_peered":"2024-09-11T00:21:28.243482+0000","last_clean":"2024-09-11T00:21:28.243482+0000","last_became_active":"2024-09-11T00:20:24.035358+0000","last_became_peered":"2024-09-11T00:20:24.035358+0000","last_unstale":"2024-09-11T00:21:28.243482+0000","last_undegraded":"2024-09-11T00:21:28.243482+0000","last_fullsized":"2024-09-11T00:21:28.243482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:56.890058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.226561+0000","last_change":"2024-09-11T00:21:28.226415+0000","last_active":"2024-09-11T00:21:28.226561+0000","last_peered":"2024-09-11T00:21:28.226561+0000","last_clean":"2024-09-11T00:21:28.226561+0000","last_became_active":"2024-09-11T00:21:28.225894+0000","last_became_peered":"2024-09-11T00:21:28.225894+0000","last_unstale":"2024-09-11T00:21:28.226561+0000","last_undegraded":"2024-09-11T00:21:28.226561+0000","last_fullsized":"2024-09-11T00:21:28.226561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:59.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228198+0000","last_change":"2024-09-11T00:20:27.068303+0000","last_active":"2024-09-11T00:21:28.228198+0000","last_peered":"2024-09-11T00:21:28.228198+0000","last_clean":"2024-09-11T00:21:28.228198+0000","last_became_active":"2024-09-11T00:20:27.066800+0000","last_became_peered":"2024-09-11T00:20:27.066800+0000","last_unstale":"2024-09-11T00:21:28.228198+0000","last_undegraded":"2024-09-11T00:21:28.228198+0000","last_fullsized":"2024-09-11T00:21:28.228198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:14.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220851+0000","last_change":"2024-09-11T00:20:24.535701+0000","last_active":"2024-09-11T00:21:28.220851+0000","last_peered":"2024-09-11T00:21:28.220851+0000","last_clean":"2024-09-11T00:21:28.220851+0000","last_became_active":"2024-09-11T00:20:24.535389+0000","last_became_peered":"2024-09-11T00:20:24.535389+0000","last_unstale":"2024-09-11T00:21:28.220851+0000","last_undegraded":"2024-09-11T00:21:28.220851+0000","last_fullsized":"2024-09-11T00:21:28.220851+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:29.044747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230045+0000","last_change":"2024-09-11T00:20:37.435584+0000","last_active":"2024-09-11T00:21:28.230045+0000","last_peered":"2024-09-11T00:21:28.230045+0000","last_clean":"2024-09-11T00:21:28.230045+0000","last_became_active":"2024-09-11T00:20:37.435237+0000","last_became_peered":"2024-09-11T00:20:37.435237+0000","last_unstale":"2024-09-11T00:21:28.230045+0000","last_undegraded":"2024-09-11T00:21:28.230045+0000","last_fullsized":"2024-09-11T00:21:28.230045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:34.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220917+0000","last_change":"2024-09-11T00:20:27.068718+0000","last_active":"2024-09-11T00:21:28.220917+0000","last_peered":"2024-09-11T00:21:28.220917+0000","last_clean":"2024-09-11T00:21:28.220917+0000","last_became_active":"2024-09-11T00:20:27.058921+0000","last_became_peered":"2024-09-11T00:20:27.058921+0000","last_unstale":"2024-09-11T00:21:28.220917+0000","last_undegraded":"2024-09-11T00:21:28.220917+0000","last_fullsized":"2024-09-11T00:21:28.220917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:16.056742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:40.069291+0000","last_change":"2024-09-11T00:20:24.023580+0000","last_active":"2024-09-11T00:21:40.069291+0000","last_peered":"2024-09-11T00:21:40.069291+0000","last_clean":"2024-09-11T00:21:40.069291+0000","last_became_active":"2024-09-11T00:20:24.023029+0000","last_became_peered":"2024-09-11T00:20:24.023029+0000","last_unstale":"2024-09-11T00:21:40.069291+0000","last_undegraded":"2024-09-11T00:21:40.069291+0000","last_fullsized":"2024-09-11T00:21:40.069291+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:05.574228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.219901+0000","last_change":"2024-09-11T00:20:37.449229+0000","last_active":"2024-09-11T00:21:28.219901+0000","last_peered":"2024-09-11T00:21:28.219901+0000","last_clean":"2024-09-11T00:21:28.219901+0000","last_became_active":"2024-09-11T00:20:37.447464+0000","last_became_peered":"2024-09-11T00:20:37.447464+0000","last_unstale":"2024-09-11T00:21:28.219901+0000","last_undegraded":"2024-09-11T00:21:28.219901+0000","last_fullsized":"2024-09-11T00:21:28.219901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:47.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.222871+0000","last_change":"2024-09-11T00:20:27.063518+0000","last_active":"2024-09-11T00:21:28.222871+0000","last_peered":"2024-09-11T00:21:28.222871+0000","last_clean":"2024-09-11T00:21:28.222871+0000","last_became_active":"2024-09-11T00:20:27.063122+0000","last_became_peered":"2024-09-11T00:20:27.063122+0000","last_unstale":"2024-09-11T00:21:28.222871+0000","last_undegraded":"2024-09-11T00:21:28.222871+0000","last_fullsized":"2024-09-11T00:21:28.222871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:52.269862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224027+0000","last_change":"2024-09-11T00:20:24.035835+0000","last_active":"2024-09-11T00:21:28.224027+0000","last_peered":"2024-09-11T00:21:28.224027+0000","last_clean":"2024-09-11T00:21:28.224027+0000","last_became_active":"2024-09-11T00:20:24.035471+0000","last_became_peered":"2024-09-11T00:20:24.035471+0000","last_unstale":"2024-09-11T00:21:28.224027+0000","last_undegraded":"2024-09-11T00:21:28.224027+0000","last_fullsized":"2024-09-11T00:21:28.224027+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:37.114863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225079+0000","last_change":"2024-09-11T00:20:37.439609+0000","last_active":"2024-09-11T00:21:28.225079+0000","last_peered":"2024-09-11T00:21:28.225079+0000","last_clean":"2024-09-11T00:21:28.225079+0000","last_became_active":"2024-09-11T00:20:37.439139+0000","last_became_peered":"2024-09-11T00:20:37.439139+0000","last_unstale":"2024-09-11T00:21:28.225079+0000","last_undegraded":"2024-09-11T00:21:28.225079+0000","last_fullsized":"2024-09-11T00:21:28.225079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:19.152096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223625+0000","last_change":"2024-09-11T00:20:27.062879+0000","last_active":"2024-09-11T00:21:28.223625+0000","last_peered":"2024-09-11T00:21:28.223625+0000","last_clean":"2024-09-11T00:21:28.223625+0000","last_became_active":"2024-09-11T00:20:27.062200+0000","last_became_peered":"2024-09-11T00:20:27.062200+0000","last_unstale":"2024-09-11T00:21:28.223625+0000","last_undegraded":"2024-09-11T00:21:28.223625+0000","last_fullsized":"2024-09-11T00:21:28.223625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:57.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230901+0000","last_change":"2024-09-11T00:21:28.230802+0000","last_active":"2024-09-11T00:21:28.230901+0000","last_peered":"2024-09-11T00:21:28.230901+0000","last_clean":"2024-09-11T00:21:28.230901+0000","last_became_active":"2024-09-11T00:21:28.230378+0000","last_became_peered":"2024-09-11T00:21:28.230378+0000","last_unstale":"2024-09-11T00:21:28.230901+0000","last_undegraded":"2024-09-11T00:21:28.230901+0000","last_fullsized":"2024-09-11T00:21:28.230901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:35.676964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223202+0000","last_change":"2024-09-11T00:20:37.444912+0000","last_active":"2024-09-11T00:21:28.223202+0000","last_peered":"2024-09-11T00:21:28.223202+0000","last_clean":"2024-09-11T00:21:28.223202+0000","last_became_active":"2024-09-11T00:20:37.444472+0000","last_became_peered":"2024-09-11T00:20:37.444472+0000","last_unstale":"2024-09-11T00:21:28.223202+0000","last_undegraded":"2024-09-11T00:21:28.223202+0000","last_fullsized":"2024-09-11T00:21:28.223202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:23.263981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223609+0000","last_change":"2024-09-11T00:20:27.060477+0000","last_active":"2024-09-11T00:21:28.223609+0000","last_peered":"2024-09-11T00:21:28.223609+0000","last_clean":"2024-09-11T00:21:28.223609+0000","last_became_active":"2024-09-11T00:20:27.060128+0000","last_became_peered":"2024-09-11T00:20:27.060128+0000","last_unstale":"2024-09-11T00:21:28.223609+0000","last_undegraded":"2024-09-11T00:21:28.223609+0000","last_fullsized":"2024-09-11T00:21:28.223609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:55.967799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244750+0000","last_change":"2024-09-11T00:20:24.028229+0000","last_active":"2024-09-11T00:21:28.244750+0000","last_peered":"2024-09-11T00:21:28.244750+0000","last_clean":"2024-09-11T00:21:28.244750+0000","last_became_active":"2024-09-11T00:20:24.027832+0000","last_became_peered":"2024-09-11T00:20:24.027832+0000","last_unstale":"2024-09-11T00:21:28.244750+0000","last_undegraded":"2024-09-11T00:21:28.244750+0000","last_fullsized":"2024-09-11T00:21:28.244750+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:19.960403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.222635+0000","last_change":"2024-09-11T00:20:37.442414+0000","last_active":"2024-09-11T00:21:28.222635+0000","last_peered":"2024-09-11T00:21:28.222635+0000","last_clean":"2024-09-11T00:21:28.222635+0000","last_became_active":"2024-09-11T00:20:37.442021+0000","last_became_peered":"2024-09-11T00:20:37.442021+0000","last_unstale":"2024-09-11T00:21:28.222635+0000","last_undegraded":"2024-09-11T00:21:28.222635+0000","last_fullsized":"2024-09-11T00:21:28.222635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:21.296638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.233392+0000","last_change":"2024-09-11T00:21:28.233303+0000","last_active":"2024-09-11T00:21:28.233392+0000","last_peered":"2024-09-11T00:21:28.233392+0000","last_clean":"2024-09-11T00:21:28.233392+0000","last_became_active":"2024-09-11T00:21:28.232039+0000","last_became_peered":"2024-09-11T00:21:28.232039+0000","last_unstale":"2024-09-11T00:21:28.233392+0000","last_undegraded":"2024-09-11T00:21:28.233392+0000","last_fullsized":"2024-09-11T00:21:28.233392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:36.828102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:39.240298+0000","last_change":"2024-09-11T00:20:24.037218+0000","last_active":"2024-09-11T00:21:39.240298+0000","last_peered":"2024-09-11T00:21:39.240298+0000","last_clean":"2024-09-11T00:21:39.240298+0000","last_became_active":"2024-09-11T00:20:24.036005+0000","last_became_peered":"2024-09-11T00:20:24.036005+0000","last_unstale":"2024-09-11T00:21:39.240298+0000","last_undegraded":"2024-09-11T00:21:39.240298+0000","last_fullsized":"2024-09-11T00:21:39.240298+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:17.758812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221624+0000","last_change":"2024-09-11T00:20:37.445584+0000","last_active":"2024-09-11T00:21:28.221624+0000","last_peered":"2024-09-11T00:21:28.221624+0000","last_clean":"2024-09-11T00:21:28.221624+0000","last_became_active":"2024-09-11T00:20:37.445196+0000","last_became_peered":"2024-09-11T00:20:37.445196+0000","last_unstale":"2024-09-11T00:21:28.221624+0000","last_undegraded":"2024-09-11T00:21:28.221624+0000","last_fullsized":"2024-09-11T00:21:28.221624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:05.966282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221404+0000","last_change":"2024-09-11T00:20:27.060428+0000","last_active":"2024-09-11T00:21:28.221404+0000","last_peered":"2024-09-11T00:21:28.221404+0000","last_clean":"2024-09-11T00:21:28.221404+0000","last_became_active":"2024-09-11T00:20:27.060057+0000","last_became_peered":"2024-09-11T00:20:27.060057+0000","last_unstale":"2024-09-11T00:21:28.221404+0000","last_undegraded":"2024-09-11T00:21:28.221404+0000","last_fullsized":"2024-09-11T00:21:28.221404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:37.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244211+0000","last_change":"2024-09-11T00:20:24.025496+0000","last_active":"2024-09-11T00:21:28.244211+0000","last_peered":"2024-09-11T00:21:28.244211+0000","last_clean":"2024-09-11T00:21:28.244211+0000","last_became_active":"2024-09-11T00:20:24.025158+0000","last_became_peered":"2024-09-11T00:20:24.025158+0000","last_unstale":"2024-09-11T00:21:28.244211+0000","last_undegraded":"2024-09-11T00:21:28.244211+0000","last_fullsized":"2024-09-11T00:21:28.244211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:40.439708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230311+0000","last_change":"2024-09-11T00:20:37.453118+0000","last_active":"2024-09-11T00:21:28.230311+0000","last_peered":"2024-09-11T00:21:28.230311+0000","last_clean":"2024-09-11T00:21:28.230311+0000","last_became_active":"2024-09-11T00:20:37.452669+0000","last_became_peered":"2024-09-11T00:20:37.452669+0000","last_unstale":"2024-09-11T00:21:28.230311+0000","last_undegraded":"2024-09-11T00:21:28.230311+0000","last_fullsized":"2024-09-11T00:21:28.230311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:51:58.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220232+0000","last_change":"2024-09-11T00:20:27.069708+0000","last_active":"2024-09-11T00:21:28.220232+0000","last_peered":"2024-09-11T00:21:28.220232+0000","last_clean":"2024-09-11T00:21:28.220232+0000","last_became_active":"2024-09-11T00:20:27.067500+0000","last_became_peered":"2024-09-11T00:20:27.067500+0000","last_unstale":"2024-09-11T00:21:28.220232+0000","last_undegraded":"2024-09-11T00:21:28.220232+0000","last_fullsized":"2024-09-11T00:21:28.220232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:57.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221600+0000","last_change":"2024-09-11T00:20:24.023504+0000","last_active":"2024-09-11T00:21:28.221600+0000","last_peered":"2024-09-11T00:21:28.221600+0000","last_clean":"2024-09-11T00:21:28.221600+0000","last_became_active":"2024-09-11T00:20:24.022706+0000","last_became_peered":"2024-09-11T00:20:24.022706+0000","last_unstale":"2024-09-11T00:21:28.221600+0000","last_undegraded":"2024-09-11T00:21:28.221600+0000","last_fullsized":"2024-09-11T00:21:28.221600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:21.216325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244655+0000","last_change":"2024-09-11T00:19:05.531191+0000","last_active":"2024-09-11T00:21:28.244655+0000","last_peered":"2024-09-11T00:21:28.244655+0000","last_clean":"2024-09-11T00:21:28.244655+0000","last_became_active":"2024-09-11T00:19:05.525664+0000","last_became_peered":"2024-09-11T00:19:05.525664+0000","last_unstale":"2024-09-11T00:21:28.244655+0000","last_undegraded":"2024-09-11T00:21:28.244655+0000","last_fullsized":"2024-09-11T00:21:28.244655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_clean_scrub_stamp":"2024-09-11T00:15:23.365345+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:06.253537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244283+0000","last_change":"2024-09-11T00:20:37.446403+0000","last_active":"2024-09-11T00:21:28.244283+0000","last_peered":"2024-09-11T00:21:28.244283+0000","last_clean":"2024-09-11T00:21:28.244283+0000","last_became_active":"2024-09-11T00:20:37.446029+0000","last_became_peered":"2024-09-11T00:20:37.446029+0000","last_unstale":"2024-09-11T00:21:28.244283+0000","last_undegraded":"2024-09-11T00:21:28.244283+0000","last_fullsized":"2024-09-11T00:21:28.244283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244424+0000","last_change":"2024-09-11T00:20:27.063086+0000","last_active":"2024-09-11T00:21:28.244424+0000","last_peered":"2024-09-11T00:21:28.244424+0000","last_clean":"2024-09-11T00:21:28.244424+0000","last_became_active":"2024-09-11T00:20:27.062790+0000","last_became_peered":"2024-09-11T00:20:27.062790+0000","last_unstale":"2024-09-11T00:21:28.244424+0000","last_undegraded":"2024-09-11T00:21:28.244424+0000","last_fullsized":"2024-09-11T00:21:28.244424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:36.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228892+0000","last_change":"2024-09-11T00:20:24.535837+0000","last_active":"2024-09-11T00:21:28.228892+0000","last_peered":"2024-09-11T00:21:28.228892+0000","last_clean":"2024-09-11T00:21:28.228892+0000","last_became_active":"2024-09-11T00:20:24.535308+0000","last_became_peered":"2024-09-11T00:20:24.535308+0000","last_unstale":"2024-09-11T00:21:28.228892+0000","last_undegraded":"2024-09-11T00:21:28.228892+0000","last_fullsized":"2024-09-11T00:21:28.228892+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.807570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243706+0000","last_change":"2024-09-11T00:20:37.443844+0000","last_active":"2024-09-11T00:21:28.243706+0000","last_peered":"2024-09-11T00:21:28.243706+0000","last_clean":"2024-09-11T00:21:28.243706+0000","last_became_active":"2024-09-11T00:20:37.443374+0000","last_became_peered":"2024-09-11T00:20:37.443374+0000","last_unstale":"2024-09-11T00:21:28.243706+0000","last_undegraded":"2024-09-11T00:21:28.243706+0000","last_fullsized":"2024-09-11T00:21:28.243706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:47.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244101+0000","last_change":"2024-09-11T00:20:27.072874+0000","last_active":"2024-09-11T00:21:28.244101+0000","last_peered":"2024-09-11T00:21:28.244101+0000","last_clean":"2024-09-11T00:21:28.244101+0000","last_became_active":"2024-09-11T00:20:27.072454+0000","last_became_peered":"2024-09-11T00:20:27.072454+0000","last_unstale":"2024-09-11T00:21:28.244101+0000","last_undegraded":"2024-09-11T00:21:28.244101+0000","last_fullsized":"2024-09-11T00:21:28.244101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:25.731037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230071+0000","last_change":"2024-09-11T00:20:24.018929+0000","last_active":"2024-09-11T00:21:28.230071+0000","last_peered":"2024-09-11T00:21:28.230071+0000","last_clean":"2024-09-11T00:21:28.230071+0000","last_became_active":"2024-09-11T00:20:24.018575+0000","last_became_peered":"2024-09-11T00:20:24.018575+0000","last_unstale":"2024-09-11T00:21:28.230071+0000","last_undegraded":"2024-09-11T00:21:28.230071+0000","last_fullsized":"2024-09-11T00:21:28.230071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.695561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228505+0000","last_change":"2024-09-11T00:20:37.453468+0000","last_active":"2024-09-11T00:21:28.228505+0000","last_peered":"2024-09-11T00:21:28.228505+0000","last_clean":"2024-09-11T00:21:28.228505+0000","last_became_active":"2024-09-11T00:20:37.453007+0000","last_became_peered":"2024-09-11T00:20:37.453007+0000","last_unstale":"2024-09-11T00:21:28.228505+0000","last_undegraded":"2024-09-11T00:21:28.228505+0000","last_fullsized":"2024-09-11T00:21:28.228505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:22.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244409+0000","last_change":"2024-09-11T00:20:27.064618+0000","last_active":"2024-09-11T00:21:28.244409+0000","last_peered":"2024-09-11T00:21:28.244409+0000","last_clean":"2024-09-11T00:21:28.244409+0000","last_became_active":"2024-09-11T00:20:27.064310+0000","last_became_peered":"2024-09-11T00:20:27.064310+0000","last_unstale":"2024-09-11T00:21:28.244409+0000","last_undegraded":"2024-09-11T00:21:28.244409+0000","last_fullsized":"2024-09-11T00:21:28.244409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:41.320561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221621+0000","last_change":"2024-09-11T00:20:24.023522+0000","last_active":"2024-09-11T00:21:28.221621+0000","last_peered":"2024-09-11T00:21:28.221621+0000","last_clean":"2024-09-11T00:21:28.221621+0000","last_became_active":"2024-09-11T00:20:24.022694+0000","last_became_peered":"2024-09-11T00:20:24.022694+0000","last_unstale":"2024-09-11T00:21:28.221621+0000","last_undegraded":"2024-09-11T00:21:28.221621+0000","last_fullsized":"2024-09-11T00:21:28.221621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:12.374179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227331+0000","last_change":"2024-09-11T00:20:37.441239+0000","last_active":"2024-09-11T00:21:28.227331+0000","last_peered":"2024-09-11T00:21:28.227331+0000","last_clean":"2024-09-11T00:21:28.227331+0000","last_became_active":"2024-09-11T00:20:37.440825+0000","last_became_peered":"2024-09-11T00:20:37.440825+0000","last_unstale":"2024-09-11T00:21:28.227331+0000","last_undegraded":"2024-09-11T00:21:28.227331+0000","last_fullsized":"2024-09-11T00:21:28.227331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:04.972569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225403+0000","last_change":"2024-09-11T00:20:27.059780+0000","last_active":"2024-09-11T00:21:28.225403+0000","last_peered":"2024-09-11T00:21:28.225403+0000","last_clean":"2024-09-11T00:21:28.225403+0000","last_became_active":"2024-09-11T00:20:27.059411+0000","last_became_peered":"2024-09-11T00:20:27.059411+0000","last_unstale":"2024-09-11T00:21:28.225403+0000","last_undegraded":"2024-09-11T00:21:28.225403+0000","last_fullsized":"2024-09-11T00:21:28.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:42.058883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224105+0000","last_change":"2024-09-11T00:20:24.024115+0000","last_active":"2024-09-11T00:21:28.224105+0000","last_peered":"2024-09-11T00:21:28.224105+0000","last_clean":"2024-09-11T00:21:28.224105+0000","last_became_active":"2024-09-11T00:20:24.023729+0000","last_became_peered":"2024-09-11T00:20:24.023729+0000","last_unstale":"2024-09-11T00:21:28.224105+0000","last_undegraded":"2024-09-11T00:21:28.224105+0000","last_fullsized":"2024-09-11T00:21:28.224105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:15.874168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228926+0000","last_change":"2024-09-11T00:20:37.443591+0000","last_active":"2024-09-11T00:21:28.228926+0000","last_peered":"2024-09-11T00:21:28.228926+0000","last_clean":"2024-09-11T00:21:28.228926+0000","last_became_active":"2024-09-11T00:20:37.443115+0000","last_became_peered":"2024-09-11T00:20:37.443115+0000","last_unstale":"2024-09-11T00:21:28.228926+0000","last_undegraded":"2024-09-11T00:21:28.228926+0000","last_fullsized":"2024-09-11T00:21:28.228926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:34.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229157+0000","last_change":"2024-09-11T00:20:27.061676+0000","last_active":"2024-09-11T00:21:28.229157+0000","last_peered":"2024-09-11T00:21:28.229157+0000","last_clean":"2024-09-11T00:21:28.229157+0000","last_became_active":"2024-09-11T00:20:27.061372+0000","last_became_peered":"2024-09-11T00:20:27.061372+0000","last_unstale":"2024-09-11T00:21:28.229157+0000","last_undegraded":"2024-09-11T00:21:28.229157+0000","last_fullsized":"2024-09-11T00:21:28.229157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:24.296673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244648+0000","last_change":"2024-09-11T00:21:28.244227+0000","last_active":"2024-09-11T00:21:28.244648+0000","last_peered":"2024-09-11T00:21:28.244648+0000","last_clean":"2024-09-11T00:21:28.244648+0000","last_became_active":"2024-09-11T00:21:28.231547+0000","last_became_peered":"2024-09-11T00:21:28.231547+0000","last_unstale":"2024-09-11T00:21:28.244648+0000","last_undegraded":"2024-09-11T00:21:28.244648+0000","last_fullsized":"2024-09-11T00:21:28.244648+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:35.119308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221502+0000","last_change":"2024-09-11T00:20:37.454603+0000","last_active":"2024-09-11T00:21:28.221502+0000","last_peered":"2024-09-11T00:21:28.221502+0000","last_clean":"2024-09-11T00:21:28.221502+0000","last_became_active":"2024-09-11T00:20:37.454123+0000","last_became_peered":"2024-09-11T00:20:37.454123+0000","last_unstale":"2024-09-11T00:21:28.221502+0000","last_undegraded":"2024-09-11T00:21:28.221502+0000","last_fullsized":"2024-09-11T00:21:28.221502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:09.111768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243988+0000","last_change":"2024-09-11T00:20:27.058806+0000","last_active":"2024-09-11T00:21:28.243988+0000","last_peered":"2024-09-11T00:21:28.243988+0000","last_clean":"2024-09-11T00:21:28.243988+0000","last_became_active":"2024-09-11T00:20:27.058458+0000","last_became_peered":"2024-09-11T00:20:27.058458+0000","last_unstale":"2024-09-11T00:21:28.243988+0000","last_undegraded":"2024-09-11T00:21:28.243988+0000","last_fullsized":"2024-09-11T00:21:28.243988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:48.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:42.246015+0000","last_change":"2024-09-11T00:20:24.022328+0000","last_active":"2024-09-11T00:21:42.246015+0000","last_peered":"2024-09-11T00:21:42.246015+0000","last_clean":"2024-09-11T00:21:42.246015+0000","last_became_active":"2024-09-11T00:20:24.021908+0000","last_became_peered":"2024-09-11T00:20:24.021908+0000","last_unstale":"2024-09-11T00:21:42.246015+0000","last_undegraded":"2024-09-11T00:21:42.246015+0000","last_fullsized":"2024-09-11T00:21:42.246015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:35.688767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225852+0000","last_change":"2024-09-11T00:20:37.444898+0000","last_active":"2024-09-11T00:21:28.225852+0000","last_peered":"2024-09-11T00:21:28.225852+0000","last_clean":"2024-09-11T00:21:28.225852+0000","last_became_active":"2024-09-11T00:20:37.443840+0000","last_became_peered":"2024-09-11T00:20:37.443840+0000","last_unstale":"2024-09-11T00:21:28.225852+0000","last_undegraded":"2024-09-11T00:21:28.225852+0000","last_fullsized":"2024-09-11T00:21:28.225852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:58.668882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243958+0000","last_change":"2024-09-11T00:20:27.061198+0000","last_active":"2024-09-11T00:21:28.243958+0000","last_peered":"2024-09-11T00:21:28.243958+0000","last_clean":"2024-09-11T00:21:28.243958+0000","last_became_active":"2024-09-11T00:20:27.060877+0000","last_became_peered":"2024-09-11T00:20:27.060877+0000","last_unstale":"2024-09-11T00:21:28.243958+0000","last_undegraded":"2024-09-11T00:21:28.243958+0000","last_fullsized":"2024-09-11T00:21:28.243958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:03.595262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228798+0000","last_change":"2024-09-11T00:20:24.022755+0000","last_active":"2024-09-11T00:21:28.228798+0000","last_peered":"2024-09-11T00:21:28.228798+0000","last_clean":"2024-09-11T00:21:28.228798+0000","last_became_active":"2024-09-11T00:20:24.021530+0000","last_became_peered":"2024-09-11T00:20:24.021530+0000","last_unstale":"2024-09-11T00:21:28.228798+0000","last_undegraded":"2024-09-11T00:21:28.228798+0000","last_fullsized":"2024-09-11T00:21:28.228798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:33.881000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225192+0000","last_change":"2024-09-11T00:20:37.443722+0000","last_active":"2024-09-11T00:21:28.225192+0000","last_peered":"2024-09-11T00:21:28.225192+0000","last_clean":"2024-09-11T00:21:28.225192+0000","last_became_active":"2024-09-11T00:20:37.443339+0000","last_became_peered":"2024-09-11T00:20:37.443339+0000","last_unstale":"2024-09-11T00:21:28.225192+0000","last_undegraded":"2024-09-11T00:21:28.225192+0000","last_fullsized":"2024-09-11T00:21:28.225192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:30.221804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221995+0000","last_change":"2024-09-11T00:20:27.073158+0000","last_active":"2024-09-11T00:21:28.221995+0000","last_peered":"2024-09-11T00:21:28.221995+0000","last_clean":"2024-09-11T00:21:28.221995+0000","last_became_active":"2024-09-11T00:20:27.072805+0000","last_became_peered":"2024-09-11T00:20:27.072805+0000","last_unstale":"2024-09-11T00:21:28.221995+0000","last_undegraded":"2024-09-11T00:21:28.221995+0000","last_fullsized":"2024-09-11T00:21:28.221995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:35.124632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228702+0000","last_change":"2024-09-11T00:20:24.027178+0000","last_active":"2024-09-11T00:21:28.228702+0000","last_peered":"2024-09-11T00:21:28.228702+0000","last_clean":"2024-09-11T00:21:28.228702+0000","last_became_active":"2024-09-11T00:20:24.026821+0000","last_became_peered":"2024-09-11T00:20:24.026821+0000","last_unstale":"2024-09-11T00:21:28.228702+0000","last_undegraded":"2024-09-11T00:21:28.228702+0000","last_fullsized":"2024-09-11T00:21:28.228702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:38.393135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229759+0000","last_change":"2024-09-11T00:20:37.447587+0000","last_active":"2024-09-11T00:21:28.229759+0000","last_peered":"2024-09-11T00:21:28.229759+0000","last_clean":"2024-09-11T00:21:28.229759+0000","last_became_active":"2024-09-11T00:20:37.447166+0000","last_became_peered":"2024-09-11T00:20:37.447166+0000","last_unstale":"2024-09-11T00:21:28.229759+0000","last_undegraded":"2024-09-11T00:21:28.229759+0000","last_fullsized":"2024-09-11T00:21:28.229759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:47.469539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223890+0000","last_change":"2024-09-11T00:20:27.059914+0000","last_active":"2024-09-11T00:21:28.223890+0000","last_peered":"2024-09-11T00:21:28.223890+0000","last_clean":"2024-09-11T00:21:28.223890+0000","last_became_active":"2024-09-11T00:20:27.059595+0000","last_became_peered":"2024-09-11T00:20:27.059595+0000","last_unstale":"2024-09-11T00:21:28.223890+0000","last_undegraded":"2024-09-11T00:21:28.223890+0000","last_fullsized":"2024-09-11T00:21:28.223890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.360010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:43.254888+0000","last_change":"2024-09-11T00:20:24.023094+0000","last_active":"2024-09-11T00:21:43.254888+0000","last_peered":"2024-09-11T00:21:43.254888+0000","last_clean":"2024-09-11T00:21:43.254888+0000","last_became_active":"2024-09-11T00:20:24.022742+0000","last_became_peered":"2024-09-11T00:20:24.022742+0000","last_unstale":"2024-09-11T00:21:43.254888+0000","last_undegraded":"2024-09-11T00:21:43.254888+0000","last_fullsized":"2024-09-11T00:21:43.254888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:41.354435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52404,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224605+0000","last_change":"2024-09-11T00:20:37.451610+0000","last_active":"2024-09-11T00:21:28.224605+0000","last_peered":"2024-09-11T00:21:28.224605+0000","last_clean":"2024-09-11T00:21:28.224605+0000","last_became_active":"2024-09-11T00:20:37.444233+0000","last_became_peered":"2024-09-11T00:20:37.444233+0000","last_unstale":"2024-09-11T00:21:28.224605+0000","last_undegraded":"2024-09-11T00:21:28.224605+0000","last_fullsized":"2024-09-11T00:21:28.224605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:02.558457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244480+0000","last_change":"2024-09-11T00:20:27.061229+0000","last_active":"2024-09-11T00:21:28.244480+0000","last_peered":"2024-09-11T00:21:28.244480+0000","last_clean":"2024-09-11T00:21:28.244480+0000","last_became_active":"2024-09-11T00:20:27.060360+0000","last_became_peered":"2024-09-11T00:20:27.060360+0000","last_unstale":"2024-09-11T00:21:28.244480+0000","last_undegraded":"2024-09-11T00:21:28.244480+0000","last_fullsized":"2024-09-11T00:21:28.244480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.107405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229947+0000","last_change":"2024-09-11T00:20:24.036483+0000","last_active":"2024-09-11T00:21:28.229947+0000","last_peered":"2024-09-11T00:21:28.229947+0000","last_clean":"2024-09-11T00:21:28.229947+0000","last_became_active":"2024-09-11T00:20:24.035855+0000","last_became_peered":"2024-09-11T00:20:24.035855+0000","last_unstale":"2024-09-11T00:21:28.229947+0000","last_undegraded":"2024-09-11T00:21:28.229947+0000","last_fullsized":"2024-09-11T00:21:28.229947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.611169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.245768+0000","last_change":"2024-09-11T00:21:28.245665+0000","last_active":"2024-09-11T00:21:28.245768+0000","last_peered":"2024-09-11T00:21:28.245768+0000","last_clean":"2024-09-11T00:21:28.245768+0000","last_became_active":"2024-09-11T00:21:28.245221+0000","last_became_peered":"2024-09-11T00:21:28.245221+0000","last_unstale":"2024-09-11T00:21:28.245768+0000","last_undegraded":"2024-09-11T00:21:28.245768+0000","last_fullsized":"2024-09-11T00:21:28.245768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:36.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.231549+0000","last_change":"2024-09-11T00:21:28.231435+0000","last_active":"2024-09-11T00:21:28.231549+0000","last_peered":"2024-09-11T00:21:28.231549+0000","last_clean":"2024-09-11T00:21:28.231549+0000","last_became_active":"2024-09-11T00:21:28.230956+0000","last_became_peered":"2024-09-11T00:21:28.230956+0000","last_unstale":"2024-09-11T00:21:28.231549+0000","last_undegraded":"2024-09-11T00:21:28.231549+0000","last_fullsized":"2024-09-11T00:21:28.231549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:28.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244367+0000","last_change":"2024-09-11T00:20:24.023068+0000","last_active":"2024-09-11T00:21:28.244367+0000","last_peered":"2024-09-11T00:21:28.244367+0000","last_clean":"2024-09-11T00:21:28.244367+0000","last_became_active":"2024-09-11T00:20:24.022718+0000","last_became_peered":"2024-09-11T00:20:24.022718+0000","last_unstale":"2024-09-11T00:21:28.244367+0000","last_undegraded":"2024-09-11T00:21:28.244367+0000","last_fullsized":"2024-09-11T00:21:28.244367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.651536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.232571+0000","last_change":"2024-09-11T00:21:28.231887+0000","last_active":"2024-09-11T00:21:28.232571+0000","last_peered":"2024-09-11T00:21:28.232571+0000","last_clean":"2024-09-11T00:21:28.232571+0000","last_became_active":"2024-09-11T00:21:28.231456+0000","last_became_peered":"2024-09-11T00:21:28.231456+0000","last_unstale":"2024-09-11T00:21:28.232571+0000","last_undegraded":"2024-09-11T00:21:28.232571+0000","last_fullsized":"2024-09-11T00:21:28.232571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:32.712598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244957+0000","last_change":"2024-09-11T00:20:27.065109+0000","last_active":"2024-09-11T00:21:28.244957+0000","last_peered":"2024-09-11T00:21:28.244957+0000","last_clean":"2024-09-11T00:21:28.244957+0000","last_became_active":"2024-09-11T00:20:27.064677+0000","last_became_peered":"2024-09-11T00:20:27.064677+0000","last_unstale":"2024-09-11T00:21:28.244957+0000","last_undegraded":"2024-09-11T00:21:28.244957+0000","last_fullsized":"2024-09-11T00:21:28.244957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:19.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":63,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:42.248455+0000","last_change":"2024-09-11T00:20:24.025347+0000","last_active":"2024-09-11T00:21:42.248455+0000","last_peered":"2024-09-11T00:21:42.248455+0000","last_clean":"2024-09-11T00:21:42.248455+0000","last_became_active":"2024-09-11T00:20:24.024980+0000","last_became_peered":"2024-09-11T00:20:24.024980+0000","last_unstale":"2024-09-11T00:21:42.248455+0000","last_undegraded":"2024-09-11T00:21:42.248455+0000","last_fullsized":"2024-09-11T00:21:42.248455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:12.530339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220226+0000","last_change":"2024-09-11T00:20:37.440340+0000","last_active":"2024-09-11T00:21:28.220226+0000","last_peered":"2024-09-11T00:21:28.220226+0000","last_clean":"2024-09-11T00:21:28.220226+0000","last_became_active":"2024-09-11T00:20:37.439896+0000","last_became_peered":"2024-09-11T00:20:37.439896+0000","last_unstale":"2024-09-11T00:21:28.220226+0000","last_undegraded":"2024-09-11T00:21:28.220226+0000","last_fullsized":"2024-09-11T00:21:28.220226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:38.774065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227373+0000","last_change":"2024-09-11T00:20:37.445860+0000","last_active":"2024-09-11T00:21:28.227373+0000","last_peered":"2024-09-11T00:21:28.227373+0000","last_clean":"2024-09-11T00:21:28.227373+0000","last_became_active":"2024-09-11T00:20:37.445450+0000","last_became_peered":"2024-09-11T00:20:37.445450+0000","last_unstale":"2024-09-11T00:21:28.227373+0000","last_undegraded":"2024-09-11T00:21:28.227373+0000","last_fullsized":"2024-09-11T00:21:28.227373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:44.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224940+0000","last_change":"2024-09-11T00:20:27.073529+0000","last_active":"2024-09-11T00:21:28.224940+0000","last_peered":"2024-09-11T00:21:28.224940+0000","last_clean":"2024-09-11T00:21:28.224940+0000","last_became_active":"2024-09-11T00:20:27.073168+0000","last_became_peered":"2024-09-11T00:20:27.073168+0000","last_unstale":"2024-09-11T00:21:28.224940+0000","last_undegraded":"2024-09-11T00:21:28.224940+0000","last_fullsized":"2024-09-11T00:21:28.224940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:16.407543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227952+0000","last_change":"2024-09-11T00:20:24.021778+0000","last_active":"2024-09-11T00:21:28.227952+0000","last_peered":"2024-09-11T00:21:28.227952+0000","last_clean":"2024-09-11T00:21:28.227952+0000","last_became_active":"2024-09-11T00:20:24.021443+0000","last_became_peered":"2024-09-11T00:20:24.021443+0000","last_unstale":"2024-09-11T00:21:28.227952+0000","last_undegraded":"2024-09-11T00:21:28.227952+0000","last_fullsized":"2024-09-11T00:21:28.227952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225267+0000","last_change":"2024-09-11T00:20:37.444279+0000","last_active":"2024-09-11T00:21:28.225267+0000","last_peered":"2024-09-11T00:21:28.225267+0000","last_clean":"2024-09-11T00:21:28.225267+0000","last_became_active":"2024-09-11T00:20:37.443821+0000","last_became_peered":"2024-09-11T00:20:37.443821+0000","last_unstale":"2024-09-11T00:21:28.225267+0000","last_undegraded":"2024-09-11T00:21:28.225267+0000","last_fullsized":"2024-09-11T00:21:28.225267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:10.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220016+0000","last_change":"2024-09-11T00:20:27.066155+0000","last_active":"2024-09-11T00:21:28.220016+0000","last_peered":"2024-09-11T00:21:28.220016+0000","last_clean":"2024-09-11T00:21:28.220016+0000","last_became_active":"2024-09-11T00:20:27.057262+0000","last_became_peered":"2024-09-11T00:20:27.057262+0000","last_unstale":"2024-09-11T00:21:28.220016+0000","last_undegraded":"2024-09-11T00:21:28.220016+0000","last_fullsized":"2024-09-11T00:21:28.220016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:54.160215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221511+0000","last_change":"2024-09-11T00:20:24.026123+0000","last_active":"2024-09-11T00:21:28.221511+0000","last_peered":"2024-09-11T00:21:28.221511+0000","last_clean":"2024-09-11T00:21:28.221511+0000","last_became_active":"2024-09-11T00:20:24.025832+0000","last_became_peered":"2024-09-11T00:20:24.025832+0000","last_unstale":"2024-09-11T00:21:28.221511+0000","last_undegraded":"2024-09-11T00:21:28.221511+0000","last_fullsized":"2024-09-11T00:21:28.221511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:45.444840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223395+0000","last_change":"2024-09-11T00:20:24.028523+0000","last_active":"2024-09-11T00:21:28.223395+0000","last_peered":"2024-09-11T00:21:28.223395+0000","last_clean":"2024-09-11T00:21:28.223395+0000","last_became_active":"2024-09-11T00:20:24.027801+0000","last_became_peered":"2024-09-11T00:20:24.027801+0000","last_unstale":"2024-09-11T00:21:28.223395+0000","last_undegraded":"2024-09-11T00:21:28.223395+0000","last_fullsized":"2024-09-11T00:21:28.223395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:14.174983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229381+0000","last_change":"2024-09-11T00:20:27.074634+0000","last_active":"2024-09-11T00:21:28.229381+0000","last_peered":"2024-09-11T00:21:28.229381+0000","last_clean":"2024-09-11T00:21:28.229381+0000","last_became_active":"2024-09-11T00:20:27.074292+0000","last_became_peered":"2024-09-11T00:20:27.074292+0000","last_unstale":"2024-09-11T00:21:28.229381+0000","last_undegraded":"2024-09-11T00:21:28.229381+0000","last_fullsized":"2024-09-11T00:21:28.229381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:07.467262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228557+0000","last_change":"2024-09-11T00:20:37.454341+0000","last_active":"2024-09-11T00:21:28.228557+0000","last_peered":"2024-09-11T00:21:28.228557+0000","last_clean":"2024-09-11T00:21:28.228557+0000","last_became_active":"2024-09-11T00:20:37.453868+0000","last_became_peered":"2024-09-11T00:20:37.453868+0000","last_unstale":"2024-09-11T00:21:28.228557+0000","last_undegraded":"2024-09-11T00:21:28.228557+0000","last_fullsized":"2024-09-11T00:21:28.228557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.015338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:42.249141+0000","last_change":"2024-09-11T00:20:24.025184+0000","last_active":"2024-09-11T00:21:42.249141+0000","last_peered":"2024-09-11T00:21:42.249141+0000","last_clean":"2024-09-11T00:21:42.249141+0000","last_became_active":"2024-09-11T00:20:24.024670+0000","last_became_peered":"2024-09-11T00:20:24.024670+0000","last_unstale":"2024-09-11T00:21:42.249141+0000","last_undegraded":"2024-09-11T00:21:42.249141+0000","last_fullsized":"2024-09-11T00:21:42.249141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:40.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223367+0000","last_change":"2024-09-11T00:20:27.057822+0000","last_active":"2024-09-11T00:21:28.223367+0000","last_peered":"2024-09-11T00:21:28.223367+0000","last_clean":"2024-09-11T00:21:28.223367+0000","last_became_active":"2024-09-11T00:20:27.057518+0000","last_became_peered":"2024-09-11T00:20:27.057518+0000","last_unstale":"2024-09-11T00:21:28.223367+0000","last_undegraded":"2024-09-11T00:21:28.223367+0000","last_fullsized":"2024-09-11T00:21:28.223367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:00.264327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227924+0000","last_change":"2024-09-11T00:20:37.438369+0000","last_active":"2024-09-11T00:21:28.227924+0000","last_peered":"2024-09-11T00:21:28.227924+0000","last_clean":"2024-09-11T00:21:28.227924+0000","last_became_active":"2024-09-11T00:20:37.437942+0000","last_became_peered":"2024-09-11T00:20:37.437942+0000","last_unstale":"2024-09-11T00:21:28.227924+0000","last_undegraded":"2024-09-11T00:21:28.227924+0000","last_fullsized":"2024-09-11T00:21:28.227924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.410905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225893+0000","last_change":"2024-09-11T00:20:27.057449+0000","last_active":"2024-09-11T00:21:28.225893+0000","last_peered":"2024-09-11T00:21:28.225893+0000","last_clean":"2024-09-11T00:21:28.225893+0000","last_became_active":"2024-09-11T00:20:27.057146+0000","last_became_peered":"2024-09-11T00:20:27.057146+0000","last_unstale":"2024-09-11T00:21:28.225893+0000","last_undegraded":"2024-09-11T00:21:28.225893+0000","last_fullsized":"2024-09-11T00:21:28.225893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:57.895062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230209+0000","last_change":"2024-09-11T00:20:27.061515+0000","last_active":"2024-09-11T00:21:28.230209+0000","last_peered":"2024-09-11T00:21:28.230209+0000","last_clean":"2024-09-11T00:21:28.230209+0000","last_became_active":"2024-09-11T00:20:27.061175+0000","last_became_peered":"2024-09-11T00:20:27.061175+0000","last_unstale":"2024-09-11T00:21:28.230209+0000","last_undegraded":"2024-09-11T00:21:28.230209+0000","last_fullsized":"2024-09-11T00:21:28.230209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:03.112574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228480+0000","last_change":"2024-09-11T00:20:27.063015+0000","last_active":"2024-09-11T00:21:28.228480+0000","last_peered":"2024-09-11T00:21:28.228480+0000","last_clean":"2024-09-11T00:21:28.228480+0000","last_became_active":"2024-09-11T00:20:27.062689+0000","last_became_peered":"2024-09-11T00:20:27.062689+0000","last_unstale":"2024-09-11T00:21:28.228480+0000","last_undegraded":"2024-09-11T00:21:28.228480+0000","last_fullsized":"2024-09-11T00:21:28.228480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.736073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228629+0000","last_change":"2024-09-11T00:20:27.062196+0000","last_active":"2024-09-11T00:21:28.228629+0000","last_peered":"2024-09-11T00:21:28.228629+0000","last_clean":"2024-09-11T00:21:28.228629+0000","last_became_active":"2024-09-11T00:20:27.061918+0000","last_became_peered":"2024-09-11T00:20:27.061918+0000","last_unstale":"2024-09-11T00:21:28.228629+0000","last_undegraded":"2024-09-11T00:21:28.228629+0000","last_fullsized":"2024-09-11T00:21:28.228629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:45.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.222783+0000","last_change":"2024-09-11T00:20:27.059021+0000","last_active":"2024-09-11T00:21:28.222783+0000","last_peered":"2024-09-11T00:21:28.222783+0000","last_clean":"2024-09-11T00:21:28.222783+0000","last_became_active":"2024-09-11T00:20:27.058726+0000","last_became_peered":"2024-09-11T00:20:27.058726+0000","last_unstale":"2024-09-11T00:21:28.222783+0000","last_undegraded":"2024-09-11T00:21:28.222783+0000","last_fullsized":"2024-09-11T00:21:28.222783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:23.900236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225475+0000","last_change":"2024-09-11T00:20:27.057593+0000","last_active":"2024-09-11T00:21:28.225475+0000","last_peered":"2024-09-11T00:21:28.225475+0000","last_clean":"2024-09-11T00:21:28.225475+0000","last_became_active":"2024-09-11T00:20:27.057310+0000","last_became_peered":"2024-09-11T00:20:27.057310+0000","last_unstale":"2024-09-11T00:21:28.225475+0000","last_undegraded":"2024-09-11T00:21:28.225475+0000","last_fullsized":"2024-09-11T00:21:28.225475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:52.355218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228593+0000","last_change":"2024-09-11T00:20:27.061140+0000","last_active":"2024-09-11T00:21:28.228593+0000","last_peered":"2024-09-11T00:21:28.228593+0000","last_clean":"2024-09-11T00:21:28.228593+0000","last_became_active":"2024-09-11T00:20:27.060500+0000","last_became_peered":"2024-09-11T00:20:27.060500+0000","last_unstale":"2024-09-11T00:21:28.228593+0000","last_undegraded":"2024-09-11T00:21:28.228593+0000","last_fullsized":"2024-09-11T00:21:28.228593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.159068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220814+0000","last_change":"2024-09-11T00:20:27.061936+0000","last_active":"2024-09-11T00:21:28.220814+0000","last_peered":"2024-09-11T00:21:28.220814+0000","last_clean":"2024-09-11T00:21:28.220814+0000","last_became_active":"2024-09-11T00:20:27.061616+0000","last_became_peered":"2024-09-11T00:20:27.061616+0000","last_unstale":"2024-09-11T00:21:28.220814+0000","last_undegraded":"2024-09-11T00:21:28.220814+0000","last_fullsized":"2024-09-11T00:21:28.220814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:32.185634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227681+0000","last_change":"2024-09-11T00:20:27.074645+0000","last_active":"2024-09-11T00:21:28.227681+0000","last_peered":"2024-09-11T00:21:28.227681+0000","last_clean":"2024-09-11T00:21:28.227681+0000","last_became_active":"2024-09-11T00:20:27.074355+0000","last_became_peered":"2024-09-11T00:20:27.074355+0000","last_unstale":"2024-09-11T00:21:28.227681+0000","last_undegraded":"2024-09-11T00:21:28.227681+0000","last_fullsized":"2024-09-11T00:21:28.227681+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220385+0000","last_change":"2024-09-11T00:20:27.069588+0000","last_active":"2024-09-11T00:21:28.220385+0000","last_peered":"2024-09-11T00:21:28.220385+0000","last_clean":"2024-09-11T00:21:28.220385+0000","last_became_active":"2024-09-11T00:20:27.067403+0000","last_became_peered":"2024-09-11T00:20:27.067403+0000","last_unstale":"2024-09-11T00:21:28.220385+0000","last_undegraded":"2024-09-11T00:21:28.220385+0000","last_fullsized":"2024-09-11T00:21:28.220385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:29.439013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243944+0000","last_change":"2024-09-11T00:20:27.071586+0000","last_active":"2024-09-11T00:21:28.243944+0000","last_peered":"2024-09-11T00:21:28.243944+0000","last_clean":"2024-09-11T00:21:28.243944+0000","last_became_active":"2024-09-11T00:20:27.071072+0000","last_became_peered":"2024-09-11T00:20:27.071072+0000","last_unstale":"2024-09-11T00:21:28.243944+0000","last_undegraded":"2024-09-11T00:21:28.243944+0000","last_fullsized":"2024-09-11T00:21:28.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:34.133027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223716+0000","last_change":"2024-09-11T00:20:27.066683+0000","last_active":"2024-09-11T00:21:28.223716+0000","last_peered":"2024-09-11T00:21:28.223716+0000","last_clean":"2024-09-11T00:21:28.223716+0000","last_became_active":"2024-09-11T00:20:27.066355+0000","last_became_peered":"2024-09-11T00:20:27.066355+0000","last_unstale":"2024-09-11T00:21:28.223716+0000","last_undegraded":"2024-09-11T00:21:28.223716+0000","last_fullsized":"2024-09-11T00:21:28.223716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:26.868072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225205+0000","last_change":"2024-09-11T00:20:27.060546+0000","last_active":"2024-09-11T00:21:28.225205+0000","last_peered":"2024-09-11T00:21:28.225205+0000","last_clean":"2024-09-11T00:21:28.225205+0000","last_became_active":"2024-09-11T00:20:27.059619+0000","last_became_peered":"2024-09-11T00:20:27.059619+0000","last_unstale":"2024-09-11T00:21:28.225205+0000","last_undegraded":"2024-09-11T00:21:28.225205+0000","last_fullsized":"2024-09-11T00:21:28.225205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.319854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228118+0000","last_change":"2024-09-11T00:20:27.073342+0000","last_active":"2024-09-11T00:21:28.228118+0000","last_peered":"2024-09-11T00:21:28.228118+0000","last_clean":"2024-09-11T00:21:28.228118+0000","last_became_active":"2024-09-11T00:20:27.072435+0000","last_became_peered":"2024-09-11T00:20:27.072435+0000","last_unstale":"2024-09-11T00:21:28.228118+0000","last_undegraded":"2024-09-11T00:21:28.228118+0000","last_fullsized":"2024-09-11T00:21:28.228118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:15.943515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225018+0000","last_change":"2024-09-11T00:20:27.063220+0000","last_active":"2024-09-11T00:21:28.225018+0000","last_peered":"2024-09-11T00:21:28.225018+0000","last_clean":"2024-09-11T00:21:28.225018+0000","last_became_active":"2024-09-11T00:20:27.062823+0000","last_became_peered":"2024-09-11T00:20:27.062823+0000","last_unstale":"2024-09-11T00:21:28.225018+0000","last_undegraded":"2024-09-11T00:21:28.225018+0000","last_fullsized":"2024-09-11T00:21:28.225018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:22.430295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225608+0000","last_change":"2024-09-11T00:20:27.062088+0000","last_active":"2024-09-11T00:21:28.225608+0000","last_peered":"2024-09-11T00:21:28.225608+0000","last_clean":"2024-09-11T00:21:28.225608+0000","last_became_active":"2024-09-11T00:20:27.061639+0000","last_became_peered":"2024-09-11T00:20:27.061639+0000","last_unstale":"2024-09-11T00:21:28.225608+0000","last_undegraded":"2024-09-11T00:21:28.225608+0000","last_fullsized":"2024-09-11T00:21:28.225608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.965908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243353+0000","last_change":"2024-09-11T00:20:27.067170+0000","last_active":"2024-09-11T00:21:28.243353+0000","last_peered":"2024-09-11T00:21:28.243353+0000","last_clean":"2024-09-11T00:21:28.243353+0000","last_became_active":"2024-09-11T00:20:27.066851+0000","last_became_peered":"2024-09-11T00:20:27.066851+0000","last_unstale":"2024-09-11T00:21:28.243353+0000","last_undegraded":"2024-09-11T00:21:28.243353+0000","last_fullsized":"2024-09-11T00:21:28.243353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:51.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.233475+0000","last_change":"2024-09-11T00:21:28.233390+0000","last_active":"2024-09-11T00:21:28.233475+0000","last_peered":"2024-09-11T00:21:28.233475+0000","last_clean":"2024-09-11T00:21:28.233475+0000","last_became_active":"2024-09-11T00:21:28.232886+0000","last_became_peered":"2024-09-11T00:21:28.232886+0000","last_unstale":"2024-09-11T00:21:28.233475+0000","last_undegraded":"2024-09-11T00:21:28.233475+0000","last_fullsized":"2024-09-11T00:21:28.233475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:26.702069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227346+0000","last_change":"2024-09-11T00:20:27.059553+0000","last_active":"2024-09-11T00:21:28.227346+0000","last_peered":"2024-09-11T00:21:28.227346+0000","last_clean":"2024-09-11T00:21:28.227346+0000","last_became_active":"2024-09-11T00:20:27.058979+0000","last_became_peered":"2024-09-11T00:20:27.058979+0000","last_unstale":"2024-09-11T00:21:28.227346+0000","last_undegraded":"2024-09-11T00:21:28.227346+0000","last_fullsized":"2024-09-11T00:21:28.227346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:21.472828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223628+0000","last_change":"2024-09-11T00:20:27.067044+0000","last_active":"2024-09-11T00:21:28.223628+0000","last_peered":"2024-09-11T00:21:28.223628+0000","last_clean":"2024-09-11T00:21:28.223628+0000","last_became_active":"2024-09-11T00:20:27.066702+0000","last_became_peered":"2024-09-11T00:20:27.066702+0000","last_unstale":"2024-09-11T00:21:28.223628+0000","last_undegraded":"2024-09-11T00:21:28.223628+0000","last_fullsized":"2024-09-11T00:21:28.223628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:21.444797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220973+0000","last_change":"2024-09-11T00:20:27.059140+0000","last_active":"2024-09-11T00:21:28.220973+0000","last_peered":"2024-09-11T00:21:28.220973+0000","last_clean":"2024-09-11T00:21:28.220973+0000","last_became_active":"2024-09-11T00:20:27.058765+0000","last_became_peered":"2024-09-11T00:20:27.058765+0000","last_unstale":"2024-09-11T00:21:28.220973+0000","last_undegraded":"2024-09-11T00:21:28.220973+0000","last_fullsized":"2024-09-11T00:21:28.220973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:43.947411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244342+0000","last_change":"2024-09-11T00:20:27.061149+0000","last_active":"2024-09-11T00:21:28.244342+0000","last_peered":"2024-09-11T00:21:28.244342+0000","last_clean":"2024-09-11T00:21:28.244342+0000","last_became_active":"2024-09-11T00:20:27.059949+0000","last_became_peered":"2024-09-11T00:20:27.059949+0000","last_unstale":"2024-09-11T00:21:28.244342+0000","last_undegraded":"2024-09-11T00:21:28.244342+0000","last_fullsized":"2024-09-11T00:21:28.244342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:03.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.231437+0000","last_change":"2024-09-11T00:21:28.231322+0000","last_active":"2024-09-11T00:21:28.231437+0000","last_peered":"2024-09-11T00:21:28.231437+0000","last_clean":"2024-09-11T00:21:28.231437+0000","last_became_active":"2024-09-11T00:21:28.231075+0000","last_became_peered":"2024-09-11T00:21:28.231075+0000","last_unstale":"2024-09-11T00:21:28.231437+0000","last_undegraded":"2024-09-11T00:21:28.231437+0000","last_fullsized":"2024-09-11T00:21:28.231437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:50.887001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.245018+0000","last_change":"2024-09-11T00:20:27.058175+0000","last_active":"2024-09-11T00:21:28.245018+0000","last_peered":"2024-09-11T00:21:28.245018+0000","last_clean":"2024-09-11T00:21:28.245018+0000","last_became_active":"2024-09-11T00:20:27.057797+0000","last_became_peered":"2024-09-11T00:20:27.057797+0000","last_unstale":"2024-09-11T00:21:28.245018+0000","last_undegraded":"2024-09-11T00:21:28.245018+0000","last_fullsized":"2024-09-11T00:21:28.245018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:29.370400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.219989+0000","last_change":"2024-09-11T00:20:27.062024+0000","last_active":"2024-09-11T00:21:28.219989+0000","last_peered":"2024-09-11T00:21:28.219989+0000","last_clean":"2024-09-11T00:21:28.219989+0000","last_became_active":"2024-09-11T00:20:27.061763+0000","last_became_peered":"2024-09-11T00:20:27.061763+0000","last_unstale":"2024-09-11T00:21:28.219989+0000","last_undegraded":"2024-09-11T00:21:28.219989+0000","last_fullsized":"2024-09-11T00:21:28.219989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:07.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243990+0000","last_change":"2024-09-11T00:20:27.060258+0000","last_active":"2024-09-11T00:21:28.243990+0000","last_peered":"2024-09-11T00:21:28.243990+0000","last_clean":"2024-09-11T00:21:28.243990+0000","last_became_active":"2024-09-11T00:20:27.059475+0000","last_became_peered":"2024-09-11T00:20:27.059475+0000","last_unstale":"2024-09-11T00:21:28.243990+0000","last_undegraded":"2024-09-11T00:21:28.243990+0000","last_fullsized":"2024-09-11T00:21:28.243990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:27.186322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223302+0000","last_change":"2024-09-11T00:20:27.059730+0000","last_active":"2024-09-11T00:21:28.223302+0000","last_peered":"2024-09-11T00:21:28.223302+0000","last_clean":"2024-09-11T00:21:28.223302+0000","last_became_active":"2024-09-11T00:20:27.059389+0000","last_became_peered":"2024-09-11T00:20:27.059389+0000","last_unstale":"2024-09-11T00:21:28.223302+0000","last_undegraded":"2024-09-11T00:21:28.223302+0000","last_fullsized":"2024-09-11T00:21:28.223302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223465+0000","last_change":"2024-09-11T00:20:27.072765+0000","last_active":"2024-09-11T00:21:28.223465+0000","last_peered":"2024-09-11T00:21:28.223465+0000","last_clean":"2024-09-11T00:21:28.223465+0000","last_became_active":"2024-09-11T00:20:27.072392+0000","last_became_peered":"2024-09-11T00:20:27.072392+0000","last_unstale":"2024-09-11T00:21:28.223465+0000","last_undegraded":"2024-09-11T00:21:28.223465+0000","last_fullsized":"2024-09-11T00:21:28.223465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:27.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220852+0000","last_change":"2024-09-11T00:20:27.056575+0000","last_active":"2024-09-11T00:21:28.220852+0000","last_peered":"2024-09-11T00:21:28.220852+0000","last_clean":"2024-09-11T00:21:28.220852+0000","last_became_active":"2024-09-11T00:20:27.056264+0000","last_became_peered":"2024-09-11T00:20:27.056264+0000","last_unstale":"2024-09-11T00:21:28.220852+0000","last_undegraded":"2024-09-11T00:21:28.220852+0000","last_fullsized":"2024-09-11T00:21:28.220852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:39.722518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227316+0000","last_change":"2024-09-11T00:20:27.065153+0000","last_active":"2024-09-11T00:21:28.227316+0000","last_peered":"2024-09-11T00:21:28.227316+0000","last_clean":"2024-09-11T00:21:28.227316+0000","last_became_active":"2024-09-11T00:20:27.062978+0000","last_became_peered":"2024-09-11T00:20:27.062978+0000","last_unstale":"2024-09-11T00:21:28.227316+0000","last_undegraded":"2024-09-11T00:21:28.227316+0000","last_fullsized":"2024-09-11T00:21:28.227316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:37.820467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228039+0000","last_change":"2024-09-11T00:20:27.070085+0000","last_active":"2024-09-11T00:21:28.228039+0000","last_peered":"2024-09-11T00:21:28.228039+0000","last_clean":"2024-09-11T00:21:28.228039+0000","last_became_active":"2024-09-11T00:20:27.069809+0000","last_became_peered":"2024-09-11T00:20:27.069809+0000","last_unstale":"2024-09-11T00:21:28.228039+0000","last_undegraded":"2024-09-11T00:21:28.228039+0000","last_fullsized":"2024-09-11T00:21:28.228039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:01.048590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225551+0000","last_change":"2024-09-11T00:20:27.064584+0000","last_active":"2024-09-11T00:21:28.225551+0000","last_peered":"2024-09-11T00:21:28.225551+0000","last_clean":"2024-09-11T00:21:28.225551+0000","last_became_active":"2024-09-11T00:20:27.064271+0000","last_became_peered":"2024-09-11T00:20:27.064271+0000","last_unstale":"2024-09-11T00:21:28.225551+0000","last_undegraded":"2024-09-11T00:21:28.225551+0000","last_fullsized":"2024-09-11T00:21:28.225551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:53.717744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56405,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":136,"num_read_kb":126,"num_write":161,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":169215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710748,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":496344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1135904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070503,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":500564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266733,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":496276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462962,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1601536,"data_stored":1092297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":31914445},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626487,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":491644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1087885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":496294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":491838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":545699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":543958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":493032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:21:46.784 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:21:46.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:46 smithi183 ceph-mon[35934]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:21:46.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:46 smithi183 ceph-mon[35934]: pgmap v357: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-11T00:21:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:46 smithi117 ceph-mon[35525]: from='client.24895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:21:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:46 smithi117 ceph-mon[35525]: pgmap v357: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-11T00:21:47.160 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 -- 172.21.15.103:0/3798394798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa170104f90 msgr2=0x7fa170105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 --2- 172.21.15.103:0/3798394798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa170104f90 0x7fa170105370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa160009ca0 tx=0x7fa16002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 -- 172.21.15.103:0/3798394798 shutdown_connections 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 --2- 172.21.15.103:0/3798394798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa17010b320 0x7fa170111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 --2- 172.21.15.103:0/3798394798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa17010a9a0 0x7fa17010ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 --2- 172.21.15.103:0/3798394798 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa170104f90 0x7fa170105370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 -- 172.21.15.103:0/3798394798 >> 172.21.15.103:0/3798394798 conn(0x7fa170100a00 msgr2=0x7fa170102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:47.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 -- 172.21.15.103:0/3798394798 shutdown_connections 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.768+0000 7fa177a00640 1 -- 172.21.15.103:0/3798394798 wait complete. 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 Processor -- start 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 -- start start 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa170104f90 0x7fa17019b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17010a9a0 0x7fa17019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa17010b320 0x7fa1701a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa170114fa0 con 0x7fa17010a9a0 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa170114e20 con 0x7fa170104f90 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.769+0000 7fa177a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa170115120 con 0x7fa17010b320 2024-09-11T00:21:47.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa170104f90 0x7fa17019b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa170104f90 0x7fa17019b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:47502/0 (socket says 172.21.15.103:47502) 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 -- 172.21.15.103:0/4080030355 learned_addr learned my addr 172.21.15.103:0/4080030355 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa17010b320 0x7fa1701a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa16fe00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17010a9a0 0x7fa17019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 -- 172.21.15.103:0/4080030355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa17010b320 msgr2=0x7fa1701a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa17010b320 0x7fa1701a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 -- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17010a9a0 msgr2=0x7fa17019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17010a9a0 0x7fa17019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 -- 172.21.15.103:0/4080030355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1701a1500 con 0x7fa170104f90 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175a00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa17010b320 0x7fa1701a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.770+0000 7fa175000640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa170104f90 0x7fa17019b450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa160005e30 tx=0x7fa160031840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.771+0000 7fa16d600640 1 -- 172.21.15.103:0/4080030355 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa160003b70 con 0x7fa170104f90 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.771+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1701a1790 con 0x7fa170104f90 2024-09-11T00:21:47.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.771+0000 7fa16d600640 1 -- 172.21.15.103:0/4080030355 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa160003d10 con 0x7fa170104f90 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.771+0000 7fa16d600640 1 -- 172.21.15.103:0/4080030355 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16002fa30 con 0x7fa170104f90 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.771+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1701a7eb0 con 0x7fa170104f90 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.772+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa170105370 con 0x7fa170104f90 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.774+0000 7fa16d600640 1 -- 172.21.15.103:0/4080030355 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fa16002fbd0 con 0x7fa170104f90 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.775+0000 7fa16d600640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1540b5370 0x7fa1540b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.776+0000 7fa16d600640 1 -- 172.21.15.103:0/4080030355 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa160107340 con 0x7fa170104f90 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.776+0000 7fa16fe00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1540b5370 0x7fa1540b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.776+0000 7fa16fe00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1540b5370 0x7fa1540b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa1701083a0 tx=0x7fa15c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:47.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.777+0000 7fa16d600640 1 -- 172.21.15.103:0/4080030355 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa1600cdce0 con 0x7fa170104f90 2024-09-11T00:21:47.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.963+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa170063590 con 0x7fa1540b5370 2024-09-11T00:21:47.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.968+0000 7fa16d600640 1 -- 172.21.15.103:0/4080030355 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340349 (secure 0 0 0) 0x7fa170063590 con 0x7fa1540b5370 2024-09-11T00:21:47.974 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:47.976 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-11T00:21:47.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1540b5370 msgr2=0x7fa1540b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1540b5370 0x7fa1540b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa1701083a0 tx=0x7fa15c005f90 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa170104f90 msgr2=0x7fa17019b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa170104f90 0x7fa17019b450 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa160005e30 tx=0x7fa160031840 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 shutdown_connections 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1540b5370 0x7fa1540b7830 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa17010b320 0x7fa1701a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa17010a9a0 0x7fa17019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 --2- 172.21.15.103:0/4080030355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa170104f90 0x7fa17019b450 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 >> 172.21.15.103:0/4080030355 conn(0x7fa170100a00 msgr2=0x7fa170102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 shutdown_connections 2024-09-11T00:21:47.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:47.974+0000 7fa177a00640 1 -- 172.21.15.103:0/4080030355 wait complete. 2024-09-11T00:21:48.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:48 smithi103 ceph-mon[30578]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:21:48.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:48 smithi103 ceph-mon[30578]: pgmap v358: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-11T00:21:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:48 smithi183 ceph-mon[35934]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:21:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:48 smithi183 ceph-mon[35934]: pgmap v358: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-11T00:21:48.826 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-11T00:21:46.378479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647007,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":217,"num_read_kb":195,"num_write":286,"num_write_kb":2256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375416,"kb_used_data":13676,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566408,"statfs":{"total":1151940427776,"available":1151556001792,"internally_reserved":0,"allocated":14004224,"data_stored":7871735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27945,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":63,"num_write":22,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003423"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225954+0000","last_change":"2024-09-11T00:20:24.027578+0000","last_active":"2024-09-11T00:21:28.225954+0000","last_peered":"2024-09-11T00:21:28.225954+0000","last_clean":"2024-09-11T00:21:28.225954+0000","last_became_active":"2024-09-11T00:20:24.027174+0000","last_became_peered":"2024-09-11T00:20:24.027174+0000","last_unstale":"2024-09-11T00:21:28.225954+0000","last_undegraded":"2024-09-11T00:21:28.225954+0000","last_fullsized":"2024-09-11T00:21:28.225954+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:15.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220553+0000","last_change":"2024-09-11T00:20:27.064047+0000","last_active":"2024-09-11T00:21:28.220553+0000","last_peered":"2024-09-11T00:21:28.220553+0000","last_clean":"2024-09-11T00:21:28.220553+0000","last_became_active":"2024-09-11T00:20:27.063756+0000","last_became_peered":"2024-09-11T00:20:27.063756+0000","last_unstale":"2024-09-11T00:21:28.220553+0000","last_undegraded":"2024-09-11T00:21:28.220553+0000","last_fullsized":"2024-09-11T00:21:28.220553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.695888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228288+0000","last_change":"2024-09-11T00:20:37.449428+0000","last_active":"2024-09-11T00:21:28.228288+0000","last_peered":"2024-09-11T00:21:28.228288+0000","last_clean":"2024-09-11T00:21:28.228288+0000","last_became_active":"2024-09-11T00:20:37.449015+0000","last_became_peered":"2024-09-11T00:20:37.449015+0000","last_unstale":"2024-09-11T00:21:28.228288+0000","last_undegraded":"2024-09-11T00:21:28.228288+0000","last_fullsized":"2024-09-11T00:21:28.228288+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:42.983990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220947+0000","last_change":"2024-09-11T00:20:24.035966+0000","last_active":"2024-09-11T00:21:28.220947+0000","last_peered":"2024-09-11T00:21:28.220947+0000","last_clean":"2024-09-11T00:21:28.220947+0000","last_became_active":"2024-09-11T00:20:24.035663+0000","last_became_peered":"2024-09-11T00:20:24.035663+0000","last_unstale":"2024-09-11T00:21:28.220947+0000","last_undegraded":"2024-09-11T00:21:28.220947+0000","last_fullsized":"2024-09-11T00:21:28.220947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:22.529392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228283+0000","last_change":"2024-09-11T00:20:27.066470+0000","last_active":"2024-09-11T00:21:28.228283+0000","last_peered":"2024-09-11T00:21:28.228283+0000","last_clean":"2024-09-11T00:21:28.228283+0000","last_became_active":"2024-09-11T00:20:27.065424+0000","last_became_peered":"2024-09-11T00:20:27.065424+0000","last_unstale":"2024-09-11T00:21:28.228283+0000","last_undegraded":"2024-09-11T00:21:28.228283+0000","last_fullsized":"2024-09-11T00:21:28.228283+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:39.967332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220052+0000","last_change":"2024-09-11T00:20:37.453105+0000","last_active":"2024-09-11T00:21:28.220052+0000","last_peered":"2024-09-11T00:21:28.220052+0000","last_clean":"2024-09-11T00:21:28.220052+0000","last_became_active":"2024-09-11T00:20:37.452319+0000","last_became_peered":"2024-09-11T00:20:37.452319+0000","last_unstale":"2024-09-11T00:21:28.220052+0000","last_undegraded":"2024-09-11T00:21:28.220052+0000","last_fullsized":"2024-09-11T00:21:28.220052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:05.873046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224025+0000","last_change":"2024-09-11T00:20:24.022277+0000","last_active":"2024-09-11T00:21:28.224025+0000","last_peered":"2024-09-11T00:21:28.224025+0000","last_clean":"2024-09-11T00:21:28.224025+0000","last_became_active":"2024-09-11T00:20:24.021893+0000","last_became_peered":"2024-09-11T00:20:24.021893+0000","last_unstale":"2024-09-11T00:21:28.224025+0000","last_undegraded":"2024-09-11T00:21:28.224025+0000","last_fullsized":"2024-09-11T00:21:28.224025+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.994032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244372+0000","last_change":"2024-09-11T00:20:27.063368+0000","last_active":"2024-09-11T00:21:28.244372+0000","last_peered":"2024-09-11T00:21:28.244372+0000","last_clean":"2024-09-11T00:21:28.244372+0000","last_became_active":"2024-09-11T00:20:27.063051+0000","last_became_peered":"2024-09-11T00:20:27.063051+0000","last_unstale":"2024-09-11T00:21:28.244372+0000","last_undegraded":"2024-09-11T00:21:28.244372+0000","last_fullsized":"2024-09-11T00:21:28.244372+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:09.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229139+0000","last_change":"2024-09-11T00:21:28.229073+0000","last_active":"2024-09-11T00:21:28.229139+0000","last_peered":"2024-09-11T00:21:28.229139+0000","last_clean":"2024-09-11T00:21:28.229139+0000","last_became_active":"2024-09-11T00:21:28.228775+0000","last_became_peered":"2024-09-11T00:21:28.228775+0000","last_unstale":"2024-09-11T00:21:28.229139+0000","last_undegraded":"2024-09-11T00:21:28.229139+0000","last_fullsized":"2024-09-11T00:21:28.229139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.886727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223773+0000","last_change":"2024-09-11T00:20:24.022562+0000","last_active":"2024-09-11T00:21:28.223773+0000","last_peered":"2024-09-11T00:21:28.223773+0000","last_clean":"2024-09-11T00:21:28.223773+0000","last_became_active":"2024-09-11T00:20:24.022207+0000","last_became_peered":"2024-09-11T00:20:24.022207+0000","last_unstale":"2024-09-11T00:21:28.223773+0000","last_undegraded":"2024-09-11T00:21:28.223773+0000","last_fullsized":"2024-09-11T00:21:28.223773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:01.071034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229830+0000","last_change":"2024-09-11T00:20:27.062572+0000","last_active":"2024-09-11T00:21:28.229830+0000","last_peered":"2024-09-11T00:21:28.229830+0000","last_clean":"2024-09-11T00:21:28.229830+0000","last_became_active":"2024-09-11T00:20:27.062288+0000","last_became_peered":"2024-09-11T00:20:27.062288+0000","last_unstale":"2024-09-11T00:21:28.229830+0000","last_undegraded":"2024-09-11T00:21:28.229830+0000","last_fullsized":"2024-09-11T00:21:28.229830+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:37.391822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220548+0000","last_change":"2024-09-11T00:20:37.444028+0000","last_active":"2024-09-11T00:21:28.220548+0000","last_peered":"2024-09-11T00:21:28.220548+0000","last_clean":"2024-09-11T00:21:28.220548+0000","last_became_active":"2024-09-11T00:20:37.443344+0000","last_became_peered":"2024-09-11T00:20:37.443344+0000","last_unstale":"2024-09-11T00:21:28.220548+0000","last_undegraded":"2024-09-11T00:21:28.220548+0000","last_fullsized":"2024-09-11T00:21:28.220548+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.347054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244654+0000","last_change":"2024-09-11T00:20:27.060317+0000","last_active":"2024-09-11T00:21:28.244654+0000","last_peered":"2024-09-11T00:21:28.244654+0000","last_clean":"2024-09-11T00:21:28.244654+0000","last_became_active":"2024-09-11T00:20:27.059983+0000","last_became_peered":"2024-09-11T00:20:27.059983+0000","last_unstale":"2024-09-11T00:21:28.244654+0000","last_undegraded":"2024-09-11T00:21:28.244654+0000","last_fullsized":"2024-09-11T00:21:28.244654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:28.405553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223622+0000","last_change":"2024-09-11T00:20:24.025555+0000","last_active":"2024-09-11T00:21:28.223622+0000","last_peered":"2024-09-11T00:21:28.223622+0000","last_clean":"2024-09-11T00:21:28.223622+0000","last_became_active":"2024-09-11T00:20:24.025171+0000","last_became_peered":"2024-09-11T00:20:24.025171+0000","last_unstale":"2024-09-11T00:21:28.223622+0000","last_undegraded":"2024-09-11T00:21:28.223622+0000","last_fullsized":"2024-09-11T00:21:28.223622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:22.708985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220409+0000","last_change":"2024-09-11T00:20:37.442057+0000","last_active":"2024-09-11T00:21:28.220409+0000","last_peered":"2024-09-11T00:21:28.220409+0000","last_clean":"2024-09-11T00:21:28.220409+0000","last_became_active":"2024-09-11T00:20:37.441555+0000","last_became_peered":"2024-09-11T00:20:37.441555+0000","last_unstale":"2024-09-11T00:21:28.220409+0000","last_undegraded":"2024-09-11T00:21:28.220409+0000","last_fullsized":"2024-09-11T00:21:28.220409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:08.155763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220312+0000","last_change":"2024-09-11T00:20:27.071533+0000","last_active":"2024-09-11T00:21:28.220312+0000","last_peered":"2024-09-11T00:21:28.220312+0000","last_clean":"2024-09-11T00:21:28.220312+0000","last_became_active":"2024-09-11T00:20:27.068771+0000","last_became_peered":"2024-09-11T00:20:27.068771+0000","last_unstale":"2024-09-11T00:21:28.220312+0000","last_undegraded":"2024-09-11T00:21:28.220312+0000","last_fullsized":"2024-09-11T00:21:28.220312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:05.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227708+0000","last_change":"2024-09-11T00:20:24.023282+0000","last_active":"2024-09-11T00:21:28.227708+0000","last_peered":"2024-09-11T00:21:28.227708+0000","last_clean":"2024-09-11T00:21:28.227708+0000","last_became_active":"2024-09-11T00:20:24.022914+0000","last_became_peered":"2024-09-11T00:20:24.022914+0000","last_unstale":"2024-09-11T00:21:28.227708+0000","last_undegraded":"2024-09-11T00:21:28.227708+0000","last_fullsized":"2024-09-11T00:21:28.227708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:45.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220653+0000","last_change":"2024-09-11T00:20:37.457338+0000","last_active":"2024-09-11T00:21:28.220653+0000","last_peered":"2024-09-11T00:21:28.220653+0000","last_clean":"2024-09-11T00:21:28.220653+0000","last_became_active":"2024-09-11T00:20:37.456993+0000","last_became_peered":"2024-09-11T00:20:37.456993+0000","last_unstale":"2024-09-11T00:21:28.220653+0000","last_undegraded":"2024-09-11T00:21:28.220653+0000","last_fullsized":"2024-09-11T00:21:28.220653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:57.284412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225994+0000","last_change":"2024-09-11T00:20:27.062752+0000","last_active":"2024-09-11T00:21:28.225994+0000","last_peered":"2024-09-11T00:21:28.225994+0000","last_clean":"2024-09-11T00:21:28.225994+0000","last_became_active":"2024-09-11T00:20:27.061569+0000","last_became_peered":"2024-09-11T00:20:27.061569+0000","last_unstale":"2024-09-11T00:21:28.225994+0000","last_undegraded":"2024-09-11T00:21:28.225994+0000","last_fullsized":"2024-09-11T00:21:28.225994+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:30.904423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221935+0000","last_change":"2024-09-11T00:20:24.023197+0000","last_active":"2024-09-11T00:21:28.221935+0000","last_peered":"2024-09-11T00:21:28.221935+0000","last_clean":"2024-09-11T00:21:28.221935+0000","last_became_active":"2024-09-11T00:20:24.022867+0000","last_became_peered":"2024-09-11T00:20:24.022867+0000","last_unstale":"2024-09-11T00:21:28.221935+0000","last_undegraded":"2024-09-11T00:21:28.221935+0000","last_fullsized":"2024-09-11T00:21:28.221935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:37.949437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225776+0000","last_change":"2024-09-11T00:20:37.451731+0000","last_active":"2024-09-11T00:21:28.225776+0000","last_peered":"2024-09-11T00:21:28.225776+0000","last_clean":"2024-09-11T00:21:28.225776+0000","last_became_active":"2024-09-11T00:20:37.447710+0000","last_became_peered":"2024-09-11T00:20:37.447710+0000","last_unstale":"2024-09-11T00:21:28.225776+0000","last_undegraded":"2024-09-11T00:21:28.225776+0000","last_fullsized":"2024-09-11T00:21:28.225776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:11:38.471216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223166+0000","last_change":"2024-09-11T00:20:27.066021+0000","last_active":"2024-09-11T00:21:28.223166+0000","last_peered":"2024-09-11T00:21:28.223166+0000","last_clean":"2024-09-11T00:21:28.223166+0000","last_became_active":"2024-09-11T00:20:27.065652+0000","last_became_peered":"2024-09-11T00:20:27.065652+0000","last_unstale":"2024-09-11T00:21:28.223166+0000","last_undegraded":"2024-09-11T00:21:28.223166+0000","last_fullsized":"2024-09-11T00:21:28.223166+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:01.933031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243482+0000","last_change":"2024-09-11T00:20:24.035619+0000","last_active":"2024-09-11T00:21:28.243482+0000","last_peered":"2024-09-11T00:21:28.243482+0000","last_clean":"2024-09-11T00:21:28.243482+0000","last_became_active":"2024-09-11T00:20:24.035358+0000","last_became_peered":"2024-09-11T00:20:24.035358+0000","last_unstale":"2024-09-11T00:21:28.243482+0000","last_undegraded":"2024-09-11T00:21:28.243482+0000","last_fullsized":"2024-09-11T00:21:28.243482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:56.890058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.226561+0000","last_change":"2024-09-11T00:21:28.226415+0000","last_active":"2024-09-11T00:21:28.226561+0000","last_peered":"2024-09-11T00:21:28.226561+0000","last_clean":"2024-09-11T00:21:28.226561+0000","last_became_active":"2024-09-11T00:21:28.225894+0000","last_became_peered":"2024-09-11T00:21:28.225894+0000","last_unstale":"2024-09-11T00:21:28.226561+0000","last_undegraded":"2024-09-11T00:21:28.226561+0000","last_fullsized":"2024-09-11T00:21:28.226561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:59.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228198+0000","last_change":"2024-09-11T00:20:27.068303+0000","last_active":"2024-09-11T00:21:28.228198+0000","last_peered":"2024-09-11T00:21:28.228198+0000","last_clean":"2024-09-11T00:21:28.228198+0000","last_became_active":"2024-09-11T00:20:27.066800+0000","last_became_peered":"2024-09-11T00:20:27.066800+0000","last_unstale":"2024-09-11T00:21:28.228198+0000","last_undegraded":"2024-09-11T00:21:28.228198+0000","last_fullsized":"2024-09-11T00:21:28.228198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:14.625397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220851+0000","last_change":"2024-09-11T00:20:24.535701+0000","last_active":"2024-09-11T00:21:28.220851+0000","last_peered":"2024-09-11T00:21:28.220851+0000","last_clean":"2024-09-11T00:21:28.220851+0000","last_became_active":"2024-09-11T00:20:24.535389+0000","last_became_peered":"2024-09-11T00:20:24.535389+0000","last_unstale":"2024-09-11T00:21:28.220851+0000","last_undegraded":"2024-09-11T00:21:28.220851+0000","last_fullsized":"2024-09-11T00:21:28.220851+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:29.044747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230045+0000","last_change":"2024-09-11T00:20:37.435584+0000","last_active":"2024-09-11T00:21:28.230045+0000","last_peered":"2024-09-11T00:21:28.230045+0000","last_clean":"2024-09-11T00:21:28.230045+0000","last_became_active":"2024-09-11T00:20:37.435237+0000","last_became_peered":"2024-09-11T00:20:37.435237+0000","last_unstale":"2024-09-11T00:21:28.230045+0000","last_undegraded":"2024-09-11T00:21:28.230045+0000","last_fullsized":"2024-09-11T00:21:28.230045+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:34.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220917+0000","last_change":"2024-09-11T00:20:27.068718+0000","last_active":"2024-09-11T00:21:28.220917+0000","last_peered":"2024-09-11T00:21:28.220917+0000","last_clean":"2024-09-11T00:21:28.220917+0000","last_became_active":"2024-09-11T00:20:27.058921+0000","last_became_peered":"2024-09-11T00:20:27.058921+0000","last_unstale":"2024-09-11T00:21:28.220917+0000","last_undegraded":"2024-09-11T00:21:28.220917+0000","last_fullsized":"2024-09-11T00:21:28.220917+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:16.056742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:40.069291+0000","last_change":"2024-09-11T00:20:24.023580+0000","last_active":"2024-09-11T00:21:40.069291+0000","last_peered":"2024-09-11T00:21:40.069291+0000","last_clean":"2024-09-11T00:21:40.069291+0000","last_became_active":"2024-09-11T00:20:24.023029+0000","last_became_peered":"2024-09-11T00:20:24.023029+0000","last_unstale":"2024-09-11T00:21:40.069291+0000","last_undegraded":"2024-09-11T00:21:40.069291+0000","last_fullsized":"2024-09-11T00:21:40.069291+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:05.574228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.219901+0000","last_change":"2024-09-11T00:20:37.449229+0000","last_active":"2024-09-11T00:21:28.219901+0000","last_peered":"2024-09-11T00:21:28.219901+0000","last_clean":"2024-09-11T00:21:28.219901+0000","last_became_active":"2024-09-11T00:20:37.447464+0000","last_became_peered":"2024-09-11T00:20:37.447464+0000","last_unstale":"2024-09-11T00:21:28.219901+0000","last_undegraded":"2024-09-11T00:21:28.219901+0000","last_fullsized":"2024-09-11T00:21:28.219901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:47.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.222871+0000","last_change":"2024-09-11T00:20:27.063518+0000","last_active":"2024-09-11T00:21:28.222871+0000","last_peered":"2024-09-11T00:21:28.222871+0000","last_clean":"2024-09-11T00:21:28.222871+0000","last_became_active":"2024-09-11T00:20:27.063122+0000","last_became_peered":"2024-09-11T00:20:27.063122+0000","last_unstale":"2024-09-11T00:21:28.222871+0000","last_undegraded":"2024-09-11T00:21:28.222871+0000","last_fullsized":"2024-09-11T00:21:28.222871+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:52.269862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224027+0000","last_change":"2024-09-11T00:20:24.035835+0000","last_active":"2024-09-11T00:21:28.224027+0000","last_peered":"2024-09-11T00:21:28.224027+0000","last_clean":"2024-09-11T00:21:28.224027+0000","last_became_active":"2024-09-11T00:20:24.035471+0000","last_became_peered":"2024-09-11T00:20:24.035471+0000","last_unstale":"2024-09-11T00:21:28.224027+0000","last_undegraded":"2024-09-11T00:21:28.224027+0000","last_fullsized":"2024-09-11T00:21:28.224027+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:37.114863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225079+0000","last_change":"2024-09-11T00:20:37.439609+0000","last_active":"2024-09-11T00:21:28.225079+0000","last_peered":"2024-09-11T00:21:28.225079+0000","last_clean":"2024-09-11T00:21:28.225079+0000","last_became_active":"2024-09-11T00:20:37.439139+0000","last_became_peered":"2024-09-11T00:20:37.439139+0000","last_unstale":"2024-09-11T00:21:28.225079+0000","last_undegraded":"2024-09-11T00:21:28.225079+0000","last_fullsized":"2024-09-11T00:21:28.225079+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:19.152096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223625+0000","last_change":"2024-09-11T00:20:27.062879+0000","last_active":"2024-09-11T00:21:28.223625+0000","last_peered":"2024-09-11T00:21:28.223625+0000","last_clean":"2024-09-11T00:21:28.223625+0000","last_became_active":"2024-09-11T00:20:27.062200+0000","last_became_peered":"2024-09-11T00:20:27.062200+0000","last_unstale":"2024-09-11T00:21:28.223625+0000","last_undegraded":"2024-09-11T00:21:28.223625+0000","last_fullsized":"2024-09-11T00:21:28.223625+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:57.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230901+0000","last_change":"2024-09-11T00:21:28.230802+0000","last_active":"2024-09-11T00:21:28.230901+0000","last_peered":"2024-09-11T00:21:28.230901+0000","last_clean":"2024-09-11T00:21:28.230901+0000","last_became_active":"2024-09-11T00:21:28.230378+0000","last_became_peered":"2024-09-11T00:21:28.230378+0000","last_unstale":"2024-09-11T00:21:28.230901+0000","last_undegraded":"2024-09-11T00:21:28.230901+0000","last_fullsized":"2024-09-11T00:21:28.230901+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:35.676964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223202+0000","last_change":"2024-09-11T00:20:37.444912+0000","last_active":"2024-09-11T00:21:28.223202+0000","last_peered":"2024-09-11T00:21:28.223202+0000","last_clean":"2024-09-11T00:21:28.223202+0000","last_became_active":"2024-09-11T00:20:37.444472+0000","last_became_peered":"2024-09-11T00:20:37.444472+0000","last_unstale":"2024-09-11T00:21:28.223202+0000","last_undegraded":"2024-09-11T00:21:28.223202+0000","last_fullsized":"2024-09-11T00:21:28.223202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:23.263981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223609+0000","last_change":"2024-09-11T00:20:27.060477+0000","last_active":"2024-09-11T00:21:28.223609+0000","last_peered":"2024-09-11T00:21:28.223609+0000","last_clean":"2024-09-11T00:21:28.223609+0000","last_became_active":"2024-09-11T00:20:27.060128+0000","last_became_peered":"2024-09-11T00:20:27.060128+0000","last_unstale":"2024-09-11T00:21:28.223609+0000","last_undegraded":"2024-09-11T00:21:28.223609+0000","last_fullsized":"2024-09-11T00:21:28.223609+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:55.967799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244750+0000","last_change":"2024-09-11T00:20:24.028229+0000","last_active":"2024-09-11T00:21:28.244750+0000","last_peered":"2024-09-11T00:21:28.244750+0000","last_clean":"2024-09-11T00:21:28.244750+0000","last_became_active":"2024-09-11T00:20:24.027832+0000","last_became_peered":"2024-09-11T00:20:24.027832+0000","last_unstale":"2024-09-11T00:21:28.244750+0000","last_undegraded":"2024-09-11T00:21:28.244750+0000","last_fullsized":"2024-09-11T00:21:28.244750+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:19.960403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.222635+0000","last_change":"2024-09-11T00:20:37.442414+0000","last_active":"2024-09-11T00:21:28.222635+0000","last_peered":"2024-09-11T00:21:28.222635+0000","last_clean":"2024-09-11T00:21:28.222635+0000","last_became_active":"2024-09-11T00:20:37.442021+0000","last_became_peered":"2024-09-11T00:20:37.442021+0000","last_unstale":"2024-09-11T00:21:28.222635+0000","last_undegraded":"2024-09-11T00:21:28.222635+0000","last_fullsized":"2024-09-11T00:21:28.222635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:21.296638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.233392+0000","last_change":"2024-09-11T00:21:28.233303+0000","last_active":"2024-09-11T00:21:28.233392+0000","last_peered":"2024-09-11T00:21:28.233392+0000","last_clean":"2024-09-11T00:21:28.233392+0000","last_became_active":"2024-09-11T00:21:28.232039+0000","last_became_peered":"2024-09-11T00:21:28.232039+0000","last_unstale":"2024-09-11T00:21:28.233392+0000","last_undegraded":"2024-09-11T00:21:28.233392+0000","last_fullsized":"2024-09-11T00:21:28.233392+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:36.828102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:44.248024+0000","last_change":"2024-09-11T00:20:24.037218+0000","last_active":"2024-09-11T00:21:44.248024+0000","last_peered":"2024-09-11T00:21:44.248024+0000","last_clean":"2024-09-11T00:21:44.248024+0000","last_became_active":"2024-09-11T00:20:24.036005+0000","last_became_peered":"2024-09-11T00:20:24.036005+0000","last_unstale":"2024-09-11T00:21:44.248024+0000","last_undegraded":"2024-09-11T00:21:44.248024+0000","last_fullsized":"2024-09-11T00:21:44.248024+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:17.758812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221624+0000","last_change":"2024-09-11T00:20:37.445584+0000","last_active":"2024-09-11T00:21:28.221624+0000","last_peered":"2024-09-11T00:21:28.221624+0000","last_clean":"2024-09-11T00:21:28.221624+0000","last_became_active":"2024-09-11T00:20:37.445196+0000","last_became_peered":"2024-09-11T00:20:37.445196+0000","last_unstale":"2024-09-11T00:21:28.221624+0000","last_undegraded":"2024-09-11T00:21:28.221624+0000","last_fullsized":"2024-09-11T00:21:28.221624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:05.966282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221404+0000","last_change":"2024-09-11T00:20:27.060428+0000","last_active":"2024-09-11T00:21:28.221404+0000","last_peered":"2024-09-11T00:21:28.221404+0000","last_clean":"2024-09-11T00:21:28.221404+0000","last_became_active":"2024-09-11T00:20:27.060057+0000","last_became_peered":"2024-09-11T00:20:27.060057+0000","last_unstale":"2024-09-11T00:21:28.221404+0000","last_undegraded":"2024-09-11T00:21:28.221404+0000","last_fullsized":"2024-09-11T00:21:28.221404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:37.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244211+0000","last_change":"2024-09-11T00:20:24.025496+0000","last_active":"2024-09-11T00:21:28.244211+0000","last_peered":"2024-09-11T00:21:28.244211+0000","last_clean":"2024-09-11T00:21:28.244211+0000","last_became_active":"2024-09-11T00:20:24.025158+0000","last_became_peered":"2024-09-11T00:20:24.025158+0000","last_unstale":"2024-09-11T00:21:28.244211+0000","last_undegraded":"2024-09-11T00:21:28.244211+0000","last_fullsized":"2024-09-11T00:21:28.244211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:40.439708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230311+0000","last_change":"2024-09-11T00:20:37.453118+0000","last_active":"2024-09-11T00:21:28.230311+0000","last_peered":"2024-09-11T00:21:28.230311+0000","last_clean":"2024-09-11T00:21:28.230311+0000","last_became_active":"2024-09-11T00:20:37.452669+0000","last_became_peered":"2024-09-11T00:20:37.452669+0000","last_unstale":"2024-09-11T00:21:28.230311+0000","last_undegraded":"2024-09-11T00:21:28.230311+0000","last_fullsized":"2024-09-11T00:21:28.230311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:51:58.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220232+0000","last_change":"2024-09-11T00:20:27.069708+0000","last_active":"2024-09-11T00:21:28.220232+0000","last_peered":"2024-09-11T00:21:28.220232+0000","last_clean":"2024-09-11T00:21:28.220232+0000","last_became_active":"2024-09-11T00:20:27.067500+0000","last_became_peered":"2024-09-11T00:20:27.067500+0000","last_unstale":"2024-09-11T00:21:28.220232+0000","last_undegraded":"2024-09-11T00:21:28.220232+0000","last_fullsized":"2024-09-11T00:21:28.220232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:57.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221600+0000","last_change":"2024-09-11T00:20:24.023504+0000","last_active":"2024-09-11T00:21:28.221600+0000","last_peered":"2024-09-11T00:21:28.221600+0000","last_clean":"2024-09-11T00:21:28.221600+0000","last_became_active":"2024-09-11T00:20:24.022706+0000","last_became_peered":"2024-09-11T00:20:24.022706+0000","last_unstale":"2024-09-11T00:21:28.221600+0000","last_undegraded":"2024-09-11T00:21:28.221600+0000","last_fullsized":"2024-09-11T00:21:28.221600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:21.216325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244655+0000","last_change":"2024-09-11T00:19:05.531191+0000","last_active":"2024-09-11T00:21:28.244655+0000","last_peered":"2024-09-11T00:21:28.244655+0000","last_clean":"2024-09-11T00:21:28.244655+0000","last_became_active":"2024-09-11T00:19:05.525664+0000","last_became_peered":"2024-09-11T00:19:05.525664+0000","last_unstale":"2024-09-11T00:21:28.244655+0000","last_undegraded":"2024-09-11T00:21:28.244655+0000","last_fullsized":"2024-09-11T00:21:28.244655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_clean_scrub_stamp":"2024-09-11T00:15:23.365345+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:06.253537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244283+0000","last_change":"2024-09-11T00:20:37.446403+0000","last_active":"2024-09-11T00:21:28.244283+0000","last_peered":"2024-09-11T00:21:28.244283+0000","last_clean":"2024-09-11T00:21:28.244283+0000","last_became_active":"2024-09-11T00:20:37.446029+0000","last_became_peered":"2024-09-11T00:20:37.446029+0000","last_unstale":"2024-09-11T00:21:28.244283+0000","last_undegraded":"2024-09-11T00:21:28.244283+0000","last_fullsized":"2024-09-11T00:21:28.244283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244424+0000","last_change":"2024-09-11T00:20:27.063086+0000","last_active":"2024-09-11T00:21:28.244424+0000","last_peered":"2024-09-11T00:21:28.244424+0000","last_clean":"2024-09-11T00:21:28.244424+0000","last_became_active":"2024-09-11T00:20:27.062790+0000","last_became_peered":"2024-09-11T00:20:27.062790+0000","last_unstale":"2024-09-11T00:21:28.244424+0000","last_undegraded":"2024-09-11T00:21:28.244424+0000","last_fullsized":"2024-09-11T00:21:28.244424+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:36.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228892+0000","last_change":"2024-09-11T00:20:24.535837+0000","last_active":"2024-09-11T00:21:28.228892+0000","last_peered":"2024-09-11T00:21:28.228892+0000","last_clean":"2024-09-11T00:21:28.228892+0000","last_became_active":"2024-09-11T00:20:24.535308+0000","last_became_peered":"2024-09-11T00:20:24.535308+0000","last_unstale":"2024-09-11T00:21:28.228892+0000","last_undegraded":"2024-09-11T00:21:28.228892+0000","last_fullsized":"2024-09-11T00:21:28.228892+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.807570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243706+0000","last_change":"2024-09-11T00:20:37.443844+0000","last_active":"2024-09-11T00:21:28.243706+0000","last_peered":"2024-09-11T00:21:28.243706+0000","last_clean":"2024-09-11T00:21:28.243706+0000","last_became_active":"2024-09-11T00:20:37.443374+0000","last_became_peered":"2024-09-11T00:20:37.443374+0000","last_unstale":"2024-09-11T00:21:28.243706+0000","last_undegraded":"2024-09-11T00:21:28.243706+0000","last_fullsized":"2024-09-11T00:21:28.243706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:47.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244101+0000","last_change":"2024-09-11T00:20:27.072874+0000","last_active":"2024-09-11T00:21:28.244101+0000","last_peered":"2024-09-11T00:21:28.244101+0000","last_clean":"2024-09-11T00:21:28.244101+0000","last_became_active":"2024-09-11T00:20:27.072454+0000","last_became_peered":"2024-09-11T00:20:27.072454+0000","last_unstale":"2024-09-11T00:21:28.244101+0000","last_undegraded":"2024-09-11T00:21:28.244101+0000","last_fullsized":"2024-09-11T00:21:28.244101+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:25.731037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230071+0000","last_change":"2024-09-11T00:20:24.018929+0000","last_active":"2024-09-11T00:21:28.230071+0000","last_peered":"2024-09-11T00:21:28.230071+0000","last_clean":"2024-09-11T00:21:28.230071+0000","last_became_active":"2024-09-11T00:20:24.018575+0000","last_became_peered":"2024-09-11T00:20:24.018575+0000","last_unstale":"2024-09-11T00:21:28.230071+0000","last_undegraded":"2024-09-11T00:21:28.230071+0000","last_fullsized":"2024-09-11T00:21:28.230071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.695561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228505+0000","last_change":"2024-09-11T00:20:37.453468+0000","last_active":"2024-09-11T00:21:28.228505+0000","last_peered":"2024-09-11T00:21:28.228505+0000","last_clean":"2024-09-11T00:21:28.228505+0000","last_became_active":"2024-09-11T00:20:37.453007+0000","last_became_peered":"2024-09-11T00:20:37.453007+0000","last_unstale":"2024-09-11T00:21:28.228505+0000","last_undegraded":"2024-09-11T00:21:28.228505+0000","last_fullsized":"2024-09-11T00:21:28.228505+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:22.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244409+0000","last_change":"2024-09-11T00:20:27.064618+0000","last_active":"2024-09-11T00:21:28.244409+0000","last_peered":"2024-09-11T00:21:28.244409+0000","last_clean":"2024-09-11T00:21:28.244409+0000","last_became_active":"2024-09-11T00:20:27.064310+0000","last_became_peered":"2024-09-11T00:20:27.064310+0000","last_unstale":"2024-09-11T00:21:28.244409+0000","last_undegraded":"2024-09-11T00:21:28.244409+0000","last_fullsized":"2024-09-11T00:21:28.244409+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:41.320561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221621+0000","last_change":"2024-09-11T00:20:24.023522+0000","last_active":"2024-09-11T00:21:28.221621+0000","last_peered":"2024-09-11T00:21:28.221621+0000","last_clean":"2024-09-11T00:21:28.221621+0000","last_became_active":"2024-09-11T00:20:24.022694+0000","last_became_peered":"2024-09-11T00:20:24.022694+0000","last_unstale":"2024-09-11T00:21:28.221621+0000","last_undegraded":"2024-09-11T00:21:28.221621+0000","last_fullsized":"2024-09-11T00:21:28.221621+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:12.374179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227331+0000","last_change":"2024-09-11T00:20:37.441239+0000","last_active":"2024-09-11T00:21:28.227331+0000","last_peered":"2024-09-11T00:21:28.227331+0000","last_clean":"2024-09-11T00:21:28.227331+0000","last_became_active":"2024-09-11T00:20:37.440825+0000","last_became_peered":"2024-09-11T00:20:37.440825+0000","last_unstale":"2024-09-11T00:21:28.227331+0000","last_undegraded":"2024-09-11T00:21:28.227331+0000","last_fullsized":"2024-09-11T00:21:28.227331+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:04.972569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225403+0000","last_change":"2024-09-11T00:20:27.059780+0000","last_active":"2024-09-11T00:21:28.225403+0000","last_peered":"2024-09-11T00:21:28.225403+0000","last_clean":"2024-09-11T00:21:28.225403+0000","last_became_active":"2024-09-11T00:20:27.059411+0000","last_became_peered":"2024-09-11T00:20:27.059411+0000","last_unstale":"2024-09-11T00:21:28.225403+0000","last_undegraded":"2024-09-11T00:21:28.225403+0000","last_fullsized":"2024-09-11T00:21:28.225403+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:42.058883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224105+0000","last_change":"2024-09-11T00:20:24.024115+0000","last_active":"2024-09-11T00:21:28.224105+0000","last_peered":"2024-09-11T00:21:28.224105+0000","last_clean":"2024-09-11T00:21:28.224105+0000","last_became_active":"2024-09-11T00:20:24.023729+0000","last_became_peered":"2024-09-11T00:20:24.023729+0000","last_unstale":"2024-09-11T00:21:28.224105+0000","last_undegraded":"2024-09-11T00:21:28.224105+0000","last_fullsized":"2024-09-11T00:21:28.224105+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:15.874168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228926+0000","last_change":"2024-09-11T00:20:37.443591+0000","last_active":"2024-09-11T00:21:28.228926+0000","last_peered":"2024-09-11T00:21:28.228926+0000","last_clean":"2024-09-11T00:21:28.228926+0000","last_became_active":"2024-09-11T00:20:37.443115+0000","last_became_peered":"2024-09-11T00:20:37.443115+0000","last_unstale":"2024-09-11T00:21:28.228926+0000","last_undegraded":"2024-09-11T00:21:28.228926+0000","last_fullsized":"2024-09-11T00:21:28.228926+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:34.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229157+0000","last_change":"2024-09-11T00:20:27.061676+0000","last_active":"2024-09-11T00:21:28.229157+0000","last_peered":"2024-09-11T00:21:28.229157+0000","last_clean":"2024-09-11T00:21:28.229157+0000","last_became_active":"2024-09-11T00:20:27.061372+0000","last_became_peered":"2024-09-11T00:20:27.061372+0000","last_unstale":"2024-09-11T00:21:28.229157+0000","last_undegraded":"2024-09-11T00:21:28.229157+0000","last_fullsized":"2024-09-11T00:21:28.229157+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:24.296673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244648+0000","last_change":"2024-09-11T00:21:28.244227+0000","last_active":"2024-09-11T00:21:28.244648+0000","last_peered":"2024-09-11T00:21:28.244648+0000","last_clean":"2024-09-11T00:21:28.244648+0000","last_became_active":"2024-09-11T00:21:28.231547+0000","last_became_peered":"2024-09-11T00:21:28.231547+0000","last_unstale":"2024-09-11T00:21:28.244648+0000","last_undegraded":"2024-09-11T00:21:28.244648+0000","last_fullsized":"2024-09-11T00:21:28.244648+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:35.119308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221502+0000","last_change":"2024-09-11T00:20:37.454603+0000","last_active":"2024-09-11T00:21:28.221502+0000","last_peered":"2024-09-11T00:21:28.221502+0000","last_clean":"2024-09-11T00:21:28.221502+0000","last_became_active":"2024-09-11T00:20:37.454123+0000","last_became_peered":"2024-09-11T00:20:37.454123+0000","last_unstale":"2024-09-11T00:21:28.221502+0000","last_undegraded":"2024-09-11T00:21:28.221502+0000","last_fullsized":"2024-09-11T00:21:28.221502+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:09.111768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243988+0000","last_change":"2024-09-11T00:20:27.058806+0000","last_active":"2024-09-11T00:21:28.243988+0000","last_peered":"2024-09-11T00:21:28.243988+0000","last_clean":"2024-09-11T00:21:28.243988+0000","last_became_active":"2024-09-11T00:20:27.058458+0000","last_became_peered":"2024-09-11T00:20:27.058458+0000","last_unstale":"2024-09-11T00:21:28.243988+0000","last_undegraded":"2024-09-11T00:21:28.243988+0000","last_fullsized":"2024-09-11T00:21:28.243988+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:48.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":65,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:42.246015+0000","last_change":"2024-09-11T00:20:24.022328+0000","last_active":"2024-09-11T00:21:42.246015+0000","last_peered":"2024-09-11T00:21:42.246015+0000","last_clean":"2024-09-11T00:21:42.246015+0000","last_became_active":"2024-09-11T00:20:24.021908+0000","last_became_peered":"2024-09-11T00:20:24.021908+0000","last_unstale":"2024-09-11T00:21:42.246015+0000","last_undegraded":"2024-09-11T00:21:42.246015+0000","last_fullsized":"2024-09-11T00:21:42.246015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:35.688767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225852+0000","last_change":"2024-09-11T00:20:37.444898+0000","last_active":"2024-09-11T00:21:28.225852+0000","last_peered":"2024-09-11T00:21:28.225852+0000","last_clean":"2024-09-11T00:21:28.225852+0000","last_became_active":"2024-09-11T00:20:37.443840+0000","last_became_peered":"2024-09-11T00:20:37.443840+0000","last_unstale":"2024-09-11T00:21:28.225852+0000","last_undegraded":"2024-09-11T00:21:28.225852+0000","last_fullsized":"2024-09-11T00:21:28.225852+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:58.668882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243958+0000","last_change":"2024-09-11T00:20:27.061198+0000","last_active":"2024-09-11T00:21:28.243958+0000","last_peered":"2024-09-11T00:21:28.243958+0000","last_clean":"2024-09-11T00:21:28.243958+0000","last_became_active":"2024-09-11T00:20:27.060877+0000","last_became_peered":"2024-09-11T00:20:27.060877+0000","last_unstale":"2024-09-11T00:21:28.243958+0000","last_undegraded":"2024-09-11T00:21:28.243958+0000","last_fullsized":"2024-09-11T00:21:28.243958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:03.595262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228798+0000","last_change":"2024-09-11T00:20:24.022755+0000","last_active":"2024-09-11T00:21:28.228798+0000","last_peered":"2024-09-11T00:21:28.228798+0000","last_clean":"2024-09-11T00:21:28.228798+0000","last_became_active":"2024-09-11T00:20:24.021530+0000","last_became_peered":"2024-09-11T00:20:24.021530+0000","last_unstale":"2024-09-11T00:21:28.228798+0000","last_undegraded":"2024-09-11T00:21:28.228798+0000","last_fullsized":"2024-09-11T00:21:28.228798+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:33.881000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225192+0000","last_change":"2024-09-11T00:20:37.443722+0000","last_active":"2024-09-11T00:21:28.225192+0000","last_peered":"2024-09-11T00:21:28.225192+0000","last_clean":"2024-09-11T00:21:28.225192+0000","last_became_active":"2024-09-11T00:20:37.443339+0000","last_became_peered":"2024-09-11T00:20:37.443339+0000","last_unstale":"2024-09-11T00:21:28.225192+0000","last_undegraded":"2024-09-11T00:21:28.225192+0000","last_fullsized":"2024-09-11T00:21:28.225192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:30.221804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221995+0000","last_change":"2024-09-11T00:20:27.073158+0000","last_active":"2024-09-11T00:21:28.221995+0000","last_peered":"2024-09-11T00:21:28.221995+0000","last_clean":"2024-09-11T00:21:28.221995+0000","last_became_active":"2024-09-11T00:20:27.072805+0000","last_became_peered":"2024-09-11T00:20:27.072805+0000","last_unstale":"2024-09-11T00:21:28.221995+0000","last_undegraded":"2024-09-11T00:21:28.221995+0000","last_fullsized":"2024-09-11T00:21:28.221995+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:35.124632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":47,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228702+0000","last_change":"2024-09-11T00:20:24.027178+0000","last_active":"2024-09-11T00:21:28.228702+0000","last_peered":"2024-09-11T00:21:28.228702+0000","last_clean":"2024-09-11T00:21:28.228702+0000","last_became_active":"2024-09-11T00:20:24.026821+0000","last_became_peered":"2024-09-11T00:20:24.026821+0000","last_unstale":"2024-09-11T00:21:28.228702+0000","last_undegraded":"2024-09-11T00:21:28.228702+0000","last_fullsized":"2024-09-11T00:21:28.228702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:38.393135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229759+0000","last_change":"2024-09-11T00:20:37.447587+0000","last_active":"2024-09-11T00:21:28.229759+0000","last_peered":"2024-09-11T00:21:28.229759+0000","last_clean":"2024-09-11T00:21:28.229759+0000","last_became_active":"2024-09-11T00:20:37.447166+0000","last_became_peered":"2024-09-11T00:20:37.447166+0000","last_unstale":"2024-09-11T00:21:28.229759+0000","last_undegraded":"2024-09-11T00:21:28.229759+0000","last_fullsized":"2024-09-11T00:21:28.229759+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:47.469539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223890+0000","last_change":"2024-09-11T00:20:27.059914+0000","last_active":"2024-09-11T00:21:28.223890+0000","last_peered":"2024-09-11T00:21:28.223890+0000","last_clean":"2024-09-11T00:21:28.223890+0000","last_became_active":"2024-09-11T00:20:27.059595+0000","last_became_peered":"2024-09-11T00:20:27.059595+0000","last_unstale":"2024-09-11T00:21:28.223890+0000","last_undegraded":"2024-09-11T00:21:28.223890+0000","last_fullsized":"2024-09-11T00:21:28.223890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.360010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:43.254888+0000","last_change":"2024-09-11T00:20:24.023094+0000","last_active":"2024-09-11T00:21:43.254888+0000","last_peered":"2024-09-11T00:21:43.254888+0000","last_clean":"2024-09-11T00:21:43.254888+0000","last_became_active":"2024-09-11T00:20:24.022742+0000","last_became_peered":"2024-09-11T00:20:24.022742+0000","last_unstale":"2024-09-11T00:21:43.254888+0000","last_undegraded":"2024-09-11T00:21:43.254888+0000","last_fullsized":"2024-09-11T00:21:43.254888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:41.354435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52404,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224605+0000","last_change":"2024-09-11T00:20:37.451610+0000","last_active":"2024-09-11T00:21:28.224605+0000","last_peered":"2024-09-11T00:21:28.224605+0000","last_clean":"2024-09-11T00:21:28.224605+0000","last_became_active":"2024-09-11T00:20:37.444233+0000","last_became_peered":"2024-09-11T00:20:37.444233+0000","last_unstale":"2024-09-11T00:21:28.224605+0000","last_undegraded":"2024-09-11T00:21:28.224605+0000","last_fullsized":"2024-09-11T00:21:28.224605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:02.558457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244480+0000","last_change":"2024-09-11T00:20:27.061229+0000","last_active":"2024-09-11T00:21:28.244480+0000","last_peered":"2024-09-11T00:21:28.244480+0000","last_clean":"2024-09-11T00:21:28.244480+0000","last_became_active":"2024-09-11T00:20:27.060360+0000","last_became_peered":"2024-09-11T00:20:27.060360+0000","last_unstale":"2024-09-11T00:21:28.244480+0000","last_undegraded":"2024-09-11T00:21:28.244480+0000","last_fullsized":"2024-09-11T00:21:28.244480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.107405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229947+0000","last_change":"2024-09-11T00:20:24.036483+0000","last_active":"2024-09-11T00:21:28.229947+0000","last_peered":"2024-09-11T00:21:28.229947+0000","last_clean":"2024-09-11T00:21:28.229947+0000","last_became_active":"2024-09-11T00:20:24.035855+0000","last_became_peered":"2024-09-11T00:20:24.035855+0000","last_unstale":"2024-09-11T00:21:28.229947+0000","last_undegraded":"2024-09-11T00:21:28.229947+0000","last_fullsized":"2024-09-11T00:21:28.229947+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.611169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.245768+0000","last_change":"2024-09-11T00:21:28.245665+0000","last_active":"2024-09-11T00:21:28.245768+0000","last_peered":"2024-09-11T00:21:28.245768+0000","last_clean":"2024-09-11T00:21:28.245768+0000","last_became_active":"2024-09-11T00:21:28.245221+0000","last_became_peered":"2024-09-11T00:21:28.245221+0000","last_unstale":"2024-09-11T00:21:28.245768+0000","last_undegraded":"2024-09-11T00:21:28.245768+0000","last_fullsized":"2024-09-11T00:21:28.245768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:36.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.231549+0000","last_change":"2024-09-11T00:21:28.231435+0000","last_active":"2024-09-11T00:21:28.231549+0000","last_peered":"2024-09-11T00:21:28.231549+0000","last_clean":"2024-09-11T00:21:28.231549+0000","last_became_active":"2024-09-11T00:21:28.230956+0000","last_became_peered":"2024-09-11T00:21:28.230956+0000","last_unstale":"2024-09-11T00:21:28.231549+0000","last_undegraded":"2024-09-11T00:21:28.231549+0000","last_fullsized":"2024-09-11T00:21:28.231549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:28.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244367+0000","last_change":"2024-09-11T00:20:24.023068+0000","last_active":"2024-09-11T00:21:28.244367+0000","last_peered":"2024-09-11T00:21:28.244367+0000","last_clean":"2024-09-11T00:21:28.244367+0000","last_became_active":"2024-09-11T00:20:24.022718+0000","last_became_peered":"2024-09-11T00:20:24.022718+0000","last_unstale":"2024-09-11T00:21:28.244367+0000","last_undegraded":"2024-09-11T00:21:28.244367+0000","last_fullsized":"2024-09-11T00:21:28.244367+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.651536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.232571+0000","last_change":"2024-09-11T00:21:28.231887+0000","last_active":"2024-09-11T00:21:28.232571+0000","last_peered":"2024-09-11T00:21:28.232571+0000","last_clean":"2024-09-11T00:21:28.232571+0000","last_became_active":"2024-09-11T00:21:28.231456+0000","last_became_peered":"2024-09-11T00:21:28.231456+0000","last_unstale":"2024-09-11T00:21:28.232571+0000","last_undegraded":"2024-09-11T00:21:28.232571+0000","last_fullsized":"2024-09-11T00:21:28.232571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:32.712598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244957+0000","last_change":"2024-09-11T00:20:27.065109+0000","last_active":"2024-09-11T00:21:28.244957+0000","last_peered":"2024-09-11T00:21:28.244957+0000","last_clean":"2024-09-11T00:21:28.244957+0000","last_became_active":"2024-09-11T00:20:27.064677+0000","last_became_peered":"2024-09-11T00:20:27.064677+0000","last_unstale":"2024-09-11T00:21:28.244957+0000","last_undegraded":"2024-09-11T00:21:28.244957+0000","last_fullsized":"2024-09-11T00:21:28.244957+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:19.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":63,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:42.248455+0000","last_change":"2024-09-11T00:20:24.025347+0000","last_active":"2024-09-11T00:21:42.248455+0000","last_peered":"2024-09-11T00:21:42.248455+0000","last_clean":"2024-09-11T00:21:42.248455+0000","last_became_active":"2024-09-11T00:20:24.024980+0000","last_became_peered":"2024-09-11T00:20:24.024980+0000","last_unstale":"2024-09-11T00:21:42.248455+0000","last_undegraded":"2024-09-11T00:21:42.248455+0000","last_fullsized":"2024-09-11T00:21:42.248455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:12.530339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220226+0000","last_change":"2024-09-11T00:20:37.440340+0000","last_active":"2024-09-11T00:21:28.220226+0000","last_peered":"2024-09-11T00:21:28.220226+0000","last_clean":"2024-09-11T00:21:28.220226+0000","last_became_active":"2024-09-11T00:20:37.439896+0000","last_became_peered":"2024-09-11T00:20:37.439896+0000","last_unstale":"2024-09-11T00:21:28.220226+0000","last_undegraded":"2024-09-11T00:21:28.220226+0000","last_fullsized":"2024-09-11T00:21:28.220226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:38.774065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227373+0000","last_change":"2024-09-11T00:20:37.445860+0000","last_active":"2024-09-11T00:21:28.227373+0000","last_peered":"2024-09-11T00:21:28.227373+0000","last_clean":"2024-09-11T00:21:28.227373+0000","last_became_active":"2024-09-11T00:20:37.445450+0000","last_became_peered":"2024-09-11T00:20:37.445450+0000","last_unstale":"2024-09-11T00:21:28.227373+0000","last_undegraded":"2024-09-11T00:21:28.227373+0000","last_fullsized":"2024-09-11T00:21:28.227373+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:44.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.224940+0000","last_change":"2024-09-11T00:20:27.073529+0000","last_active":"2024-09-11T00:21:28.224940+0000","last_peered":"2024-09-11T00:21:28.224940+0000","last_clean":"2024-09-11T00:21:28.224940+0000","last_became_active":"2024-09-11T00:20:27.073168+0000","last_became_peered":"2024-09-11T00:20:27.073168+0000","last_unstale":"2024-09-11T00:21:28.224940+0000","last_undegraded":"2024-09-11T00:21:28.224940+0000","last_fullsized":"2024-09-11T00:21:28.224940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:16.407543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227952+0000","last_change":"2024-09-11T00:20:24.021778+0000","last_active":"2024-09-11T00:21:28.227952+0000","last_peered":"2024-09-11T00:21:28.227952+0000","last_clean":"2024-09-11T00:21:28.227952+0000","last_became_active":"2024-09-11T00:20:24.021443+0000","last_became_peered":"2024-09-11T00:20:24.021443+0000","last_unstale":"2024-09-11T00:21:28.227952+0000","last_undegraded":"2024-09-11T00:21:28.227952+0000","last_fullsized":"2024-09-11T00:21:28.227952+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225267+0000","last_change":"2024-09-11T00:20:37.444279+0000","last_active":"2024-09-11T00:21:28.225267+0000","last_peered":"2024-09-11T00:21:28.225267+0000","last_clean":"2024-09-11T00:21:28.225267+0000","last_became_active":"2024-09-11T00:20:37.443821+0000","last_became_peered":"2024-09-11T00:20:37.443821+0000","last_unstale":"2024-09-11T00:21:28.225267+0000","last_undegraded":"2024-09-11T00:21:28.225267+0000","last_fullsized":"2024-09-11T00:21:28.225267+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:10.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220016+0000","last_change":"2024-09-11T00:20:27.066155+0000","last_active":"2024-09-11T00:21:28.220016+0000","last_peered":"2024-09-11T00:21:28.220016+0000","last_clean":"2024-09-11T00:21:28.220016+0000","last_became_active":"2024-09-11T00:20:27.057262+0000","last_became_peered":"2024-09-11T00:20:27.057262+0000","last_unstale":"2024-09-11T00:21:28.220016+0000","last_undegraded":"2024-09-11T00:21:28.220016+0000","last_fullsized":"2024-09-11T00:21:28.220016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:54.160215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.221511+0000","last_change":"2024-09-11T00:20:24.026123+0000","last_active":"2024-09-11T00:21:28.221511+0000","last_peered":"2024-09-11T00:21:28.221511+0000","last_clean":"2024-09-11T00:21:28.221511+0000","last_became_active":"2024-09-11T00:20:24.025832+0000","last_became_peered":"2024-09-11T00:20:24.025832+0000","last_unstale":"2024-09-11T00:21:28.221511+0000","last_undegraded":"2024-09-11T00:21:28.221511+0000","last_fullsized":"2024-09-11T00:21:28.221511+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:45.444840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223395+0000","last_change":"2024-09-11T00:20:24.028523+0000","last_active":"2024-09-11T00:21:28.223395+0000","last_peered":"2024-09-11T00:21:28.223395+0000","last_clean":"2024-09-11T00:21:28.223395+0000","last_became_active":"2024-09-11T00:20:24.027801+0000","last_became_peered":"2024-09-11T00:20:24.027801+0000","last_unstale":"2024-09-11T00:21:28.223395+0000","last_undegraded":"2024-09-11T00:21:28.223395+0000","last_fullsized":"2024-09-11T00:21:28.223395+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:14.174983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.229381+0000","last_change":"2024-09-11T00:20:27.074634+0000","last_active":"2024-09-11T00:21:28.229381+0000","last_peered":"2024-09-11T00:21:28.229381+0000","last_clean":"2024-09-11T00:21:28.229381+0000","last_became_active":"2024-09-11T00:20:27.074292+0000","last_became_peered":"2024-09-11T00:20:27.074292+0000","last_unstale":"2024-09-11T00:21:28.229381+0000","last_undegraded":"2024-09-11T00:21:28.229381+0000","last_fullsized":"2024-09-11T00:21:28.229381+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:07.467262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228557+0000","last_change":"2024-09-11T00:20:37.454341+0000","last_active":"2024-09-11T00:21:28.228557+0000","last_peered":"2024-09-11T00:21:28.228557+0000","last_clean":"2024-09-11T00:21:28.228557+0000","last_became_active":"2024-09-11T00:20:37.453868+0000","last_became_peered":"2024-09-11T00:20:37.453868+0000","last_unstale":"2024-09-11T00:21:28.228557+0000","last_undegraded":"2024-09-11T00:21:28.228557+0000","last_fullsized":"2024-09-11T00:21:28.228557+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.015338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:42.249141+0000","last_change":"2024-09-11T00:20:24.025184+0000","last_active":"2024-09-11T00:21:42.249141+0000","last_peered":"2024-09-11T00:21:42.249141+0000","last_clean":"2024-09-11T00:21:42.249141+0000","last_became_active":"2024-09-11T00:20:24.024670+0000","last_became_peered":"2024-09-11T00:20:24.024670+0000","last_unstale":"2024-09-11T00:21:42.249141+0000","last_undegraded":"2024-09-11T00:21:42.249141+0000","last_fullsized":"2024-09-11T00:21:42.249141+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:40.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223367+0000","last_change":"2024-09-11T00:20:27.057822+0000","last_active":"2024-09-11T00:21:28.223367+0000","last_peered":"2024-09-11T00:21:28.223367+0000","last_clean":"2024-09-11T00:21:28.223367+0000","last_became_active":"2024-09-11T00:20:27.057518+0000","last_became_peered":"2024-09-11T00:20:27.057518+0000","last_unstale":"2024-09-11T00:21:28.223367+0000","last_undegraded":"2024-09-11T00:21:28.223367+0000","last_fullsized":"2024-09-11T00:21:28.223367+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:00.264327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227924+0000","last_change":"2024-09-11T00:20:37.438369+0000","last_active":"2024-09-11T00:21:28.227924+0000","last_peered":"2024-09-11T00:21:28.227924+0000","last_clean":"2024-09-11T00:21:28.227924+0000","last_became_active":"2024-09-11T00:20:37.437942+0000","last_became_peered":"2024-09-11T00:20:37.437942+0000","last_unstale":"2024-09-11T00:21:28.227924+0000","last_undegraded":"2024-09-11T00:21:28.227924+0000","last_fullsized":"2024-09-11T00:21:28.227924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.410905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225893+0000","last_change":"2024-09-11T00:20:27.057449+0000","last_active":"2024-09-11T00:21:28.225893+0000","last_peered":"2024-09-11T00:21:28.225893+0000","last_clean":"2024-09-11T00:21:28.225893+0000","last_became_active":"2024-09-11T00:20:27.057146+0000","last_became_peered":"2024-09-11T00:20:27.057146+0000","last_unstale":"2024-09-11T00:21:28.225893+0000","last_undegraded":"2024-09-11T00:21:28.225893+0000","last_fullsized":"2024-09-11T00:21:28.225893+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:57.895062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.230209+0000","last_change":"2024-09-11T00:20:27.061515+0000","last_active":"2024-09-11T00:21:28.230209+0000","last_peered":"2024-09-11T00:21:28.230209+0000","last_clean":"2024-09-11T00:21:28.230209+0000","last_became_active":"2024-09-11T00:20:27.061175+0000","last_became_peered":"2024-09-11T00:20:27.061175+0000","last_unstale":"2024-09-11T00:21:28.230209+0000","last_undegraded":"2024-09-11T00:21:28.230209+0000","last_fullsized":"2024-09-11T00:21:28.230209+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:03.112574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228480+0000","last_change":"2024-09-11T00:20:27.063015+0000","last_active":"2024-09-11T00:21:28.228480+0000","last_peered":"2024-09-11T00:21:28.228480+0000","last_clean":"2024-09-11T00:21:28.228480+0000","last_became_active":"2024-09-11T00:20:27.062689+0000","last_became_peered":"2024-09-11T00:20:27.062689+0000","last_unstale":"2024-09-11T00:21:28.228480+0000","last_undegraded":"2024-09-11T00:21:28.228480+0000","last_fullsized":"2024-09-11T00:21:28.228480+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.736073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228629+0000","last_change":"2024-09-11T00:20:27.062196+0000","last_active":"2024-09-11T00:21:28.228629+0000","last_peered":"2024-09-11T00:21:28.228629+0000","last_clean":"2024-09-11T00:21:28.228629+0000","last_became_active":"2024-09-11T00:20:27.061918+0000","last_became_peered":"2024-09-11T00:20:27.061918+0000","last_unstale":"2024-09-11T00:21:28.228629+0000","last_undegraded":"2024-09-11T00:21:28.228629+0000","last_fullsized":"2024-09-11T00:21:28.228629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:45.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.222783+0000","last_change":"2024-09-11T00:20:27.059021+0000","last_active":"2024-09-11T00:21:28.222783+0000","last_peered":"2024-09-11T00:21:28.222783+0000","last_clean":"2024-09-11T00:21:28.222783+0000","last_became_active":"2024-09-11T00:20:27.058726+0000","last_became_peered":"2024-09-11T00:20:27.058726+0000","last_unstale":"2024-09-11T00:21:28.222783+0000","last_undegraded":"2024-09-11T00:21:28.222783+0000","last_fullsized":"2024-09-11T00:21:28.222783+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:23.900236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225475+0000","last_change":"2024-09-11T00:20:27.057593+0000","last_active":"2024-09-11T00:21:28.225475+0000","last_peered":"2024-09-11T00:21:28.225475+0000","last_clean":"2024-09-11T00:21:28.225475+0000","last_became_active":"2024-09-11T00:20:27.057310+0000","last_became_peered":"2024-09-11T00:20:27.057310+0000","last_unstale":"2024-09-11T00:21:28.225475+0000","last_undegraded":"2024-09-11T00:21:28.225475+0000","last_fullsized":"2024-09-11T00:21:28.225475+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:52.355218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228593+0000","last_change":"2024-09-11T00:20:27.061140+0000","last_active":"2024-09-11T00:21:28.228593+0000","last_peered":"2024-09-11T00:21:28.228593+0000","last_clean":"2024-09-11T00:21:28.228593+0000","last_became_active":"2024-09-11T00:20:27.060500+0000","last_became_peered":"2024-09-11T00:20:27.060500+0000","last_unstale":"2024-09-11T00:21:28.228593+0000","last_undegraded":"2024-09-11T00:21:28.228593+0000","last_fullsized":"2024-09-11T00:21:28.228593+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.159068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220814+0000","last_change":"2024-09-11T00:20:27.061936+0000","last_active":"2024-09-11T00:21:28.220814+0000","last_peered":"2024-09-11T00:21:28.220814+0000","last_clean":"2024-09-11T00:21:28.220814+0000","last_became_active":"2024-09-11T00:20:27.061616+0000","last_became_peered":"2024-09-11T00:20:27.061616+0000","last_unstale":"2024-09-11T00:21:28.220814+0000","last_undegraded":"2024-09-11T00:21:28.220814+0000","last_fullsized":"2024-09-11T00:21:28.220814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:32.185634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227681+0000","last_change":"2024-09-11T00:20:27.074645+0000","last_active":"2024-09-11T00:21:28.227681+0000","last_peered":"2024-09-11T00:21:28.227681+0000","last_clean":"2024-09-11T00:21:28.227681+0000","last_became_active":"2024-09-11T00:20:27.074355+0000","last_became_peered":"2024-09-11T00:20:27.074355+0000","last_unstale":"2024-09-11T00:21:28.227681+0000","last_undegraded":"2024-09-11T00:21:28.227681+0000","last_fullsized":"2024-09-11T00:21:28.227681+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220385+0000","last_change":"2024-09-11T00:20:27.069588+0000","last_active":"2024-09-11T00:21:28.220385+0000","last_peered":"2024-09-11T00:21:28.220385+0000","last_clean":"2024-09-11T00:21:28.220385+0000","last_became_active":"2024-09-11T00:20:27.067403+0000","last_became_peered":"2024-09-11T00:20:27.067403+0000","last_unstale":"2024-09-11T00:21:28.220385+0000","last_undegraded":"2024-09-11T00:21:28.220385+0000","last_fullsized":"2024-09-11T00:21:28.220385+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:29.439013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243944+0000","last_change":"2024-09-11T00:20:27.071586+0000","last_active":"2024-09-11T00:21:28.243944+0000","last_peered":"2024-09-11T00:21:28.243944+0000","last_clean":"2024-09-11T00:21:28.243944+0000","last_became_active":"2024-09-11T00:20:27.071072+0000","last_became_peered":"2024-09-11T00:20:27.071072+0000","last_unstale":"2024-09-11T00:21:28.243944+0000","last_undegraded":"2024-09-11T00:21:28.243944+0000","last_fullsized":"2024-09-11T00:21:28.243944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:34.133027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223716+0000","last_change":"2024-09-11T00:20:27.066683+0000","last_active":"2024-09-11T00:21:28.223716+0000","last_peered":"2024-09-11T00:21:28.223716+0000","last_clean":"2024-09-11T00:21:28.223716+0000","last_became_active":"2024-09-11T00:20:27.066355+0000","last_became_peered":"2024-09-11T00:20:27.066355+0000","last_unstale":"2024-09-11T00:21:28.223716+0000","last_undegraded":"2024-09-11T00:21:28.223716+0000","last_fullsized":"2024-09-11T00:21:28.223716+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:26.868072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225205+0000","last_change":"2024-09-11T00:20:27.060546+0000","last_active":"2024-09-11T00:21:28.225205+0000","last_peered":"2024-09-11T00:21:28.225205+0000","last_clean":"2024-09-11T00:21:28.225205+0000","last_became_active":"2024-09-11T00:20:27.059619+0000","last_became_peered":"2024-09-11T00:20:27.059619+0000","last_unstale":"2024-09-11T00:21:28.225205+0000","last_undegraded":"2024-09-11T00:21:28.225205+0000","last_fullsized":"2024-09-11T00:21:28.225205+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.319854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228118+0000","last_change":"2024-09-11T00:20:27.073342+0000","last_active":"2024-09-11T00:21:28.228118+0000","last_peered":"2024-09-11T00:21:28.228118+0000","last_clean":"2024-09-11T00:21:28.228118+0000","last_became_active":"2024-09-11T00:20:27.072435+0000","last_became_peered":"2024-09-11T00:20:27.072435+0000","last_unstale":"2024-09-11T00:21:28.228118+0000","last_undegraded":"2024-09-11T00:21:28.228118+0000","last_fullsized":"2024-09-11T00:21:28.228118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:15.943515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225018+0000","last_change":"2024-09-11T00:20:27.063220+0000","last_active":"2024-09-11T00:21:28.225018+0000","last_peered":"2024-09-11T00:21:28.225018+0000","last_clean":"2024-09-11T00:21:28.225018+0000","last_became_active":"2024-09-11T00:20:27.062823+0000","last_became_peered":"2024-09-11T00:20:27.062823+0000","last_unstale":"2024-09-11T00:21:28.225018+0000","last_undegraded":"2024-09-11T00:21:28.225018+0000","last_fullsized":"2024-09-11T00:21:28.225018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:22.430295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225608+0000","last_change":"2024-09-11T00:20:27.062088+0000","last_active":"2024-09-11T00:21:28.225608+0000","last_peered":"2024-09-11T00:21:28.225608+0000","last_clean":"2024-09-11T00:21:28.225608+0000","last_became_active":"2024-09-11T00:20:27.061639+0000","last_became_peered":"2024-09-11T00:20:27.061639+0000","last_unstale":"2024-09-11T00:21:28.225608+0000","last_undegraded":"2024-09-11T00:21:28.225608+0000","last_fullsized":"2024-09-11T00:21:28.225608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.965908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243353+0000","last_change":"2024-09-11T00:20:27.067170+0000","last_active":"2024-09-11T00:21:28.243353+0000","last_peered":"2024-09-11T00:21:28.243353+0000","last_clean":"2024-09-11T00:21:28.243353+0000","last_became_active":"2024-09-11T00:20:27.066851+0000","last_became_peered":"2024-09-11T00:20:27.066851+0000","last_unstale":"2024-09-11T00:21:28.243353+0000","last_undegraded":"2024-09-11T00:21:28.243353+0000","last_fullsized":"2024-09-11T00:21:28.243353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:51.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.233475+0000","last_change":"2024-09-11T00:21:28.233390+0000","last_active":"2024-09-11T00:21:28.233475+0000","last_peered":"2024-09-11T00:21:28.233475+0000","last_clean":"2024-09-11T00:21:28.233475+0000","last_became_active":"2024-09-11T00:21:28.232886+0000","last_became_peered":"2024-09-11T00:21:28.232886+0000","last_unstale":"2024-09-11T00:21:28.233475+0000","last_undegraded":"2024-09-11T00:21:28.233475+0000","last_fullsized":"2024-09-11T00:21:28.233475+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:26.702069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227346+0000","last_change":"2024-09-11T00:20:27.059553+0000","last_active":"2024-09-11T00:21:28.227346+0000","last_peered":"2024-09-11T00:21:28.227346+0000","last_clean":"2024-09-11T00:21:28.227346+0000","last_became_active":"2024-09-11T00:20:27.058979+0000","last_became_peered":"2024-09-11T00:20:27.058979+0000","last_unstale":"2024-09-11T00:21:28.227346+0000","last_undegraded":"2024-09-11T00:21:28.227346+0000","last_fullsized":"2024-09-11T00:21:28.227346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:21.472828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223628+0000","last_change":"2024-09-11T00:20:27.067044+0000","last_active":"2024-09-11T00:21:28.223628+0000","last_peered":"2024-09-11T00:21:28.223628+0000","last_clean":"2024-09-11T00:21:28.223628+0000","last_became_active":"2024-09-11T00:20:27.066702+0000","last_became_peered":"2024-09-11T00:20:27.066702+0000","last_unstale":"2024-09-11T00:21:28.223628+0000","last_undegraded":"2024-09-11T00:21:28.223628+0000","last_fullsized":"2024-09-11T00:21:28.223628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:35:21.444797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220973+0000","last_change":"2024-09-11T00:20:27.059140+0000","last_active":"2024-09-11T00:21:28.220973+0000","last_peered":"2024-09-11T00:21:28.220973+0000","last_clean":"2024-09-11T00:21:28.220973+0000","last_became_active":"2024-09-11T00:20:27.058765+0000","last_became_peered":"2024-09-11T00:20:27.058765+0000","last_unstale":"2024-09-11T00:21:28.220973+0000","last_undegraded":"2024-09-11T00:21:28.220973+0000","last_fullsized":"2024-09-11T00:21:28.220973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:43.947411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.244342+0000","last_change":"2024-09-11T00:20:27.061149+0000","last_active":"2024-09-11T00:21:28.244342+0000","last_peered":"2024-09-11T00:21:28.244342+0000","last_clean":"2024-09-11T00:21:28.244342+0000","last_became_active":"2024-09-11T00:20:27.059949+0000","last_became_peered":"2024-09-11T00:20:27.059949+0000","last_unstale":"2024-09-11T00:21:28.244342+0000","last_undegraded":"2024-09-11T00:21:28.244342+0000","last_fullsized":"2024-09-11T00:21:28.244342+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:03.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.231437+0000","last_change":"2024-09-11T00:21:28.231322+0000","last_active":"2024-09-11T00:21:28.231437+0000","last_peered":"2024-09-11T00:21:28.231437+0000","last_clean":"2024-09-11T00:21:28.231437+0000","last_became_active":"2024-09-11T00:21:28.231075+0000","last_became_peered":"2024-09-11T00:21:28.231075+0000","last_unstale":"2024-09-11T00:21:28.231437+0000","last_undegraded":"2024-09-11T00:21:28.231437+0000","last_fullsized":"2024-09-11T00:21:28.231437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:50.887001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.245018+0000","last_change":"2024-09-11T00:20:27.058175+0000","last_active":"2024-09-11T00:21:28.245018+0000","last_peered":"2024-09-11T00:21:28.245018+0000","last_clean":"2024-09-11T00:21:28.245018+0000","last_became_active":"2024-09-11T00:20:27.057797+0000","last_became_peered":"2024-09-11T00:20:27.057797+0000","last_unstale":"2024-09-11T00:21:28.245018+0000","last_undegraded":"2024-09-11T00:21:28.245018+0000","last_fullsized":"2024-09-11T00:21:28.245018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:29.370400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.219989+0000","last_change":"2024-09-11T00:20:27.062024+0000","last_active":"2024-09-11T00:21:28.219989+0000","last_peered":"2024-09-11T00:21:28.219989+0000","last_clean":"2024-09-11T00:21:28.219989+0000","last_became_active":"2024-09-11T00:20:27.061763+0000","last_became_peered":"2024-09-11T00:20:27.061763+0000","last_unstale":"2024-09-11T00:21:28.219989+0000","last_undegraded":"2024-09-11T00:21:28.219989+0000","last_fullsized":"2024-09-11T00:21:28.219989+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:07.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.243990+0000","last_change":"2024-09-11T00:20:27.060258+0000","last_active":"2024-09-11T00:21:28.243990+0000","last_peered":"2024-09-11T00:21:28.243990+0000","last_clean":"2024-09-11T00:21:28.243990+0000","last_became_active":"2024-09-11T00:20:27.059475+0000","last_became_peered":"2024-09-11T00:20:27.059475+0000","last_unstale":"2024-09-11T00:21:28.243990+0000","last_undegraded":"2024-09-11T00:21:28.243990+0000","last_fullsized":"2024-09-11T00:21:28.243990+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:27.186322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223302+0000","last_change":"2024-09-11T00:20:27.059730+0000","last_active":"2024-09-11T00:21:28.223302+0000","last_peered":"2024-09-11T00:21:28.223302+0000","last_clean":"2024-09-11T00:21:28.223302+0000","last_became_active":"2024-09-11T00:20:27.059389+0000","last_became_peered":"2024-09-11T00:20:27.059389+0000","last_unstale":"2024-09-11T00:21:28.223302+0000","last_undegraded":"2024-09-11T00:21:28.223302+0000","last_fullsized":"2024-09-11T00:21:28.223302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.223465+0000","last_change":"2024-09-11T00:20:27.072765+0000","last_active":"2024-09-11T00:21:28.223465+0000","last_peered":"2024-09-11T00:21:28.223465+0000","last_clean":"2024-09-11T00:21:28.223465+0000","last_became_active":"2024-09-11T00:20:27.072392+0000","last_became_peered":"2024-09-11T00:20:27.072392+0000","last_unstale":"2024-09-11T00:21:28.223465+0000","last_undegraded":"2024-09-11T00:21:28.223465+0000","last_fullsized":"2024-09-11T00:21:28.223465+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:27.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.220852+0000","last_change":"2024-09-11T00:20:27.056575+0000","last_active":"2024-09-11T00:21:28.220852+0000","last_peered":"2024-09-11T00:21:28.220852+0000","last_clean":"2024-09-11T00:21:28.220852+0000","last_became_active":"2024-09-11T00:20:27.056264+0000","last_became_peered":"2024-09-11T00:20:27.056264+0000","last_unstale":"2024-09-11T00:21:28.220852+0000","last_undegraded":"2024-09-11T00:21:28.220852+0000","last_fullsized":"2024-09-11T00:21:28.220852+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:39.722518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.227316+0000","last_change":"2024-09-11T00:20:27.065153+0000","last_active":"2024-09-11T00:21:28.227316+0000","last_peered":"2024-09-11T00:21:28.227316+0000","last_clean":"2024-09-11T00:21:28.227316+0000","last_became_active":"2024-09-11T00:20:27.062978+0000","last_became_peered":"2024-09-11T00:20:27.062978+0000","last_unstale":"2024-09-11T00:21:28.227316+0000","last_undegraded":"2024-09-11T00:21:28.227316+0000","last_fullsized":"2024-09-11T00:21:28.227316+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:37.820467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.228039+0000","last_change":"2024-09-11T00:20:27.070085+0000","last_active":"2024-09-11T00:21:28.228039+0000","last_peered":"2024-09-11T00:21:28.228039+0000","last_clean":"2024-09-11T00:21:28.228039+0000","last_became_active":"2024-09-11T00:20:27.069809+0000","last_became_peered":"2024-09-11T00:20:27.069809+0000","last_unstale":"2024-09-11T00:21:28.228039+0000","last_undegraded":"2024-09-11T00:21:28.228039+0000","last_fullsized":"2024-09-11T00:21:28.228039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:01.048590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-11T00:21:28.225551+0000","last_change":"2024-09-11T00:20:27.064584+0000","last_active":"2024-09-11T00:21:28.225551+0000","last_peered":"2024-09-11T00:21:28.225551+0000","last_clean":"2024-09-11T00:21:28.225551+0000","last_became_active":"2024-09-11T00:20:27.064271+0000","last_became_peered":"2024-09-11T00:20:27.064271+0000","last_unstale":"2024-09-11T00:21:28.225551+0000","last_undegraded":"2024-09-11T00:21:28.225551+0000","last_fullsized":"2024-09-11T00:21:28.225551+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:53.717744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56405,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":141,"num_read_kb":131,"num_write":161,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":169215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710749,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":496344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906977,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1135904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070503,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":500564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266733,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":496276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462962,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32748,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712404,"statfs":{"total":95995035648,"available":95961501696,"internally_reserved":0,"allocated":1601536,"data_stored":1092297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":31914445},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626488,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":491644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1609728,"data_stored":1087885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":496294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":491838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":545699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":543958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":493032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:21:48.832 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:21:48.832 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T00:21:48.833 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T00:21:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:48 smithi117 ceph-mon[35525]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:21:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:48 smithi117 ceph-mon[35525]: pgmap v358: 129 pgs: 129 active+clean; 632 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 7 op/s 2024-09-11T00:21:49.235 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:49.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.896+0000 7f23fa400640 1 -- 172.21.15.103:0/1914590734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f41065c0 msgr2=0x7f23f410cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:49.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.896+0000 7f23fa400640 1 --2- 172.21.15.103:0/1914590734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f41065c0 0x7f23f410cf40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009cd0 tx=0x7f23ec02a8e0 comp rx=0 tx=0).stop 2024-09-11T00:21:49.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.896+0000 7f23fa400640 1 -- 172.21.15.103:0/1914590734 shutdown_connections 2024-09-11T00:21:49.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.896+0000 7f23fa400640 1 --2- 172.21.15.103:0/1914590734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f41065c0 0x7f23f410cf40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:49.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.896+0000 7f23fa400640 1 --2- 172.21.15.103:0/1914590734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f4105c00 0x7f23f4106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.896+0000 7f23fa400640 1 --2- 172.21.15.103:0/1914590734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23f4104aa0 0x7f23f4104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.896+0000 7f23fa400640 1 -- 172.21.15.103:0/1914590734 >> 172.21.15.103:0/1914590734 conn(0x7f23f4100370 msgr2=0x7f23f41027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.897+0000 7f23fa400640 1 -- 172.21.15.103:0/1914590734 shutdown_connections 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.897+0000 7f23fa400640 1 -- 172.21.15.103:0/1914590734 wait complete. 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.897+0000 7f23fa400640 1 Processor -- start 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23fa400640 1 -- start start 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23fa400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23f4104aa0 0x7f23f419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23fa400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f4105c00 0x7f23f419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23fa400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f41065c0 0x7f23f419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23fa400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4110700 con 0x7f23f4105c00 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23fa400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4110580 con 0x7f23f41065c0 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23fa400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23f4110880 con 0x7f23f4104aa0 2024-09-11T00:21:49.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f4105c00 0x7f23f419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f4105c00 0x7f23f419b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35046/0 (socket says 172.21.15.103:35046) 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23ebe00640 1 -- 172.21.15.103:0/1840867332 learned_addr learned my addr 172.21.15.103:0/1840867332 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23f9000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23f4104aa0 0x7f23f419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.898+0000 7f23f9a00640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f41065c0 0x7f23f419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f9000640 1 -- 172.21.15.103:0/1840867332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f41065c0 msgr2=0x7f23f419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f9000640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f41065c0 0x7f23f419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f9000640 1 -- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f4105c00 msgr2=0x7f23f419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f9000640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f4105c00 0x7f23f419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f9000640 1 -- 172.21.15.103:0/1840867332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f419c8a0 con 0x7f23f4104aa0 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f9000640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23f4104aa0 0x7f23f419ad70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23e400e9f0 tx=0x7f23e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f2000640 1 -- 172.21.15.103:0/1840867332 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e400ccc0 con 0x7f23f4104aa0 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f419cb90 con 0x7f23f4104aa0 2024-09-11T00:21:49.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f2000640 1 -- 172.21.15.103:0/1840867332 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e40108e0 con 0x7f23f4104aa0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23f2000640 1 -- 172.21.15.103:0/1840867332 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4018880 con 0x7f23f4104aa0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.899+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f41a3270 con 0x7f23f4104aa0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.900+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f410bd50 con 0x7f23f4104aa0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.903+0000 7f23f2000640 1 -- 172.21.15.103:0/1840867332 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f23e4010430 con 0x7f23f4104aa0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.904+0000 7f23f2000640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f23d80b5410 0x7f23d80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.904+0000 7f23f2000640 1 -- 172.21.15.103:0/1840867332 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f23e40e68b0 con 0x7f23f4104aa0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.904+0000 7f23ebe00640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f23d80b5410 0x7f23d80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.905+0000 7f23ebe00640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f23d80b5410 0x7f23d80b78d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009c90 tx=0x7f23dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:49.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:49.906+0000 7f23f2000640 1 -- 172.21.15.103:0/1840867332 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f23e40acc00 con 0x7f23f4104aa0 2024-09-11T00:21:50.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.163+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f23f402e820 con 0x7f23f4104aa0 2024-09-11T00:21:50.172 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-09-11T00:21:50.172 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQCe4uBmc8PfCRAAZko4EGKc1JZIQUC0l16img== 2024-09-11T00:21:50.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.170+0000 7f23f2000640 1 -- 172.21.15.103:0/1840867332 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f23e40b1630 con 0x7f23f4104aa0 2024-09-11T00:21:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f23d80b5410 msgr2=0x7f23d80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f23d80b5410 0x7f23d80b78d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009c90 tx=0x7f23dc005f90 comp rx=0 tx=0).stop 2024-09-11T00:21:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23f4104aa0 msgr2=0x7f23f419ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:50.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23f4104aa0 0x7f23f419ad70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23e400e9f0 tx=0x7f23e400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 shutdown_connections 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f23d80b5410 0x7f23d80b78d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f41065c0 0x7f23f419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23f4105c00 0x7f23f419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 --2- 172.21.15.103:0/1840867332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23f4104aa0 0x7f23f419ad70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 >> 172.21.15.103:0/1840867332 conn(0x7f23f4100370 msgr2=0x7f23f4108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 shutdown_connections 2024-09-11T00:21:50.175 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:50.172+0000 7f23fa400640 1 -- 172.21.15.103:0/1840867332 wait complete. 2024-09-11T00:21:50.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:50 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1840867332' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:21:50.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:50 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:21:50.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:50 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:21:50.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:50 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1840867332' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:21:50.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:50 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:21:50.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:50 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:21:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:50 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1840867332' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:21:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:50 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-11T00:21:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:50 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T00:21:51.059 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:21:51.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T00:21:51.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T00:21:51.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-11T00:21:51.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:51 smithi103 ceph-mon[30578]: pgmap v359: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-11T00:21:51.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:51 smithi183 ceph-mon[35934]: pgmap v359: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-11T00:21:51.556 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:51 smithi117 ceph-mon[35525]: pgmap v359: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-11T00:21:52.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 -- 172.21.15.103:0/3656316430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8104e10 msgr2=0x7fa6f81051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 --2- 172.21.15.103:0/3656316430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8104e10 0x7fa6f81051f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec009900 tx=0x7fa6ec02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 -- 172.21.15.103:0/3656316430 shutdown_connections 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 --2- 172.21.15.103:0/3656316430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6f810b1a0 0x7fa6f81119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 --2- 172.21.15.103:0/3656316430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f810a820 0x7fa6f810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 --2- 172.21.15.103:0/3656316430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f8104e10 0x7fa6f81051f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 -- 172.21.15.103:0/3656316430 >> 172.21.15.103:0/3656316430 conn(0x7fa6f8100840 msgr2=0x7fa6f8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 -- 172.21.15.103:0/3656316430 shutdown_connections 2024-09-11T00:21:52.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.182+0000 7fa6fee00640 1 -- 172.21.15.103:0/3656316430 wait complete. 2024-09-11T00:21:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 Processor -- start 2024-09-11T00:21:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 -- start start 2024-09-11T00:21:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8104e10 0x7fa6f819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f810a820 0x7fa6f819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6f810b1a0 0x7fa6f81a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8114e60 con 0x7fa6f8104e10 2024-09-11T00:21:52.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8114ce0 con 0x7fa6f810a820 2024-09-11T00:21:52.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.184+0000 7fa6fee00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8114fe0 con 0x7fa6f810b1a0 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f810a820 0x7fa6f819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8104e10 0x7fa6f819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f810a820 0x7fa6f819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48418/0 (socket says 172.21.15.103:48418) 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 -- 172.21.15.103:0/1772715778 learned_addr learned my addr 172.21.15.103:0/1772715778 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fe400640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6f810b1a0 0x7fa6f81a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 -- 172.21.15.103:0/1772715778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6f810b1a0 msgr2=0x7fa6f81a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6f810b1a0 0x7fa6f81a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 -- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8104e10 msgr2=0x7fa6f819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8104e10 0x7fa6f819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 -- 172.21.15.103:0/1772715778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f81a12c0 con 0x7fa6f810a820 2024-09-11T00:21:52.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6fd000640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f810a820 0x7fa6f819b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400ea60 tx=0x7fa6e400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.185+0000 7fa6f6000640 1 -- 172.21.15.103:0/1772715778 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400cde0 con 0x7fa6f810a820 2024-09-11T00:21:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.186+0000 7fa6f6000640 1 -- 172.21.15.103:0/1772715778 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6e40035f0 con 0x7fa6f810a820 2024-09-11T00:21:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.186+0000 7fa6f6000640 1 -- 172.21.15.103:0/1772715778 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4010690 con 0x7fa6f810a820 2024-09-11T00:21:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.186+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f81a15b0 con 0x7fa6f810a820 2024-09-11T00:21:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.186+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f81a7c70 con 0x7fa6f810a820 2024-09-11T00:21:52.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.187+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f81051f0 con 0x7fa6f810a820 2024-09-11T00:21:52.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.190+0000 7fa6f6000640 1 -- 172.21.15.103:0/1772715778 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fa6e4004000 con 0x7fa6f810a820 2024-09-11T00:21:52.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.191+0000 7fa6f6000640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa6e00b5410 0x7fa6e00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:52.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.191+0000 7fa6f6000640 1 -- 172.21.15.103:0/1772715778 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fa6e40e6600 con 0x7fa6f810a820 2024-09-11T00:21:52.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.191+0000 7fa6fda00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa6e00b5410 0x7fa6e00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:52.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.192+0000 7fa6f6000640 1 -- 172.21.15.103:0/1772715778 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa6e4056220 con 0x7fa6f810a820 2024-09-11T00:21:52.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.192+0000 7fa6fda00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa6e00b5410 0x7fa6e00b78d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec031440 tx=0x7fa6ec005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:52.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.378+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fa6f819c340 con 0x7fa6f810a820 2024-09-11T00:21:52.381 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:52.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.379+0000 7fa6f6000640 1 -- 172.21.15.103:0/1772715778 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v49) ==== 86+0+133 (secure 0 0 0) 0x7fa6f819c340 con 0x7fa6f810a820 2024-09-11T00:21:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa6e00b5410 msgr2=0x7fa6e00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa6e00b5410 0x7fa6e00b78d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec031440 tx=0x7fa6ec005b50 comp rx=0 tx=0).stop 2024-09-11T00:21:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f810a820 msgr2=0x7fa6f819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:52.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f810a820 0x7fa6f819b750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400ea60 tx=0x7fa6e400ef20 comp rx=0 tx=0).stop 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 shutdown_connections 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa6e00b5410 0x7fa6e00b78d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa6f810b1a0 0x7fa6f81a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6f810a820 0x7fa6f819b750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 --2- 172.21.15.103:0/1772715778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f8104e10 0x7fa6f819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 >> 172.21.15.103:0/1772715778 conn(0x7fa6f8100840 msgr2=0x7fa6f8102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 shutdown_connections 2024-09-11T00:21:52.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:52.381+0000 7fa6fee00640 1 -- 172.21.15.103:0/1772715778 wait complete. 2024-09-11T00:21:52.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:52 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1772715778' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:21:52.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:52 smithi103 ceph-mon[30578]: pgmap v360: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-11T00:21:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:52 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1772715778' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:21:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:52 smithi117 ceph-mon[35525]: pgmap v360: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-11T00:21:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:52 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1772715778' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-11T00:21:52.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:52 smithi183 ceph-mon[35934]: pgmap v360: 129 pgs: 129 active+clean; 636 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-11T00:21:53.188 INFO:teuthology.orchestra.run.smithi103.stdout:[{"version":49,"timestamp":"2024-09-11T00:20:14.247983+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-11T00:21:53.189 INFO:tasks.ceph_manager:config epoch is 49 2024-09-11T00:21:53.189 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T00:21:53.189 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T00:21:53.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T00:21:53.566 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:54.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 -- 172.21.15.103:0/2564154656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102c00 msgr2=0x7f2c18102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:54.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 --2- 172.21.15.103:0/2564154656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102c00 0x7f2c18102fe0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009900 tx=0x7f2c0802a400 comp rx=0 tx=0).stop 2024-09-11T00:21:54.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 -- 172.21.15.103:0/2564154656 shutdown_connections 2024-09-11T00:21:54.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 --2- 172.21.15.103:0/2564154656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c18108f90 0x7f2c1810f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.202 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 --2- 172.21.15.103:0/2564154656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c18108610 0x7f2c18108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 --2- 172.21.15.103:0/2564154656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18102c00 0x7f2c18102fe0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 -- 172.21.15.103:0/2564154656 >> 172.21.15.103:0/2564154656 conn(0x7f2c180fe6d0 msgr2=0x7f2c18100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:54.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 -- 172.21.15.103:0/2564154656 shutdown_connections 2024-09-11T00:21:54.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.200+0000 7f2c1d800640 1 -- 172.21.15.103:0/2564154656 wait complete. 2024-09-11T00:21:54.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.201+0000 7f2c1d800640 1 Processor -- start 2024-09-11T00:21:54.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1d800640 1 -- start start 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c18102c00 0x7f2c18199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18108610 0x7f2c181995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c18108f90 0x7f2c1819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1d800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18112ca0 con 0x7f2c18108610 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18112b20 con 0x7f2c18102c00 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18112e20 con 0x7f2c18108f90 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18108610 0x7f2c181995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18108610 0x7f2c181995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40660/0 (socket says 172.21.15.103:40660) 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c0fe00640 1 -- 172.21.15.103:0/4208921949 learned_addr learned my addr 172.21.15.103:0/4208921949 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c1ce00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c18108f90 0x7f2c1819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.202+0000 7f2c17e00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c18102c00 0x7f2c18199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c1ce00640 1 -- 172.21.15.103:0/4208921949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c18102c00 msgr2=0x7f2c18199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:54.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c1ce00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c18102c00 0x7f2c18199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c1ce00640 1 -- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18108610 msgr2=0x7f2c181995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:54.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c1ce00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18108610 0x7f2c181995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c1ce00640 1 -- 172.21.15.103:0/4208921949 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1819f120 con 0x7f2c18108f90 2024-09-11T00:21:54.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c0fe00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18108610 0x7f2c181995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:54.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c17e00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c18102c00 0x7f2c18199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:54.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c1ce00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c18108f90 0x7f2c1819e950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fd0 tx=0x7f2c1000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:54.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c15600640 1 -- 172.21.15.103:0/4208921949 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000ce20 con 0x7f2c18108f90 2024-09-11T00:21:54.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1819f410 con 0x7f2c18108f90 2024-09-11T00:21:54.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c15600640 1 -- 172.21.15.103:0/4208921949 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c10005020 con 0x7f2c18108f90 2024-09-11T00:21:54.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.203+0000 7f2c15600640 1 -- 172.21.15.103:0/4208921949 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10010690 con 0x7f2c18108f90 2024-09-11T00:21:54.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.204+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181a5ad0 con 0x7f2c18108f90 2024-09-11T00:21:54.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.205+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c18102fe0 con 0x7f2c18108f90 2024-09-11T00:21:54.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.207+0000 7f2c15600640 1 -- 172.21.15.103:0/4208921949 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f2c100054d0 con 0x7f2c18108f90 2024-09-11T00:21:54.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.208+0000 7f2c15600640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2bfc0b5490 0x7f2bfc0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:54.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.209+0000 7f2c15600640 1 -- 172.21.15.103:0/4208921949 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2c100e6730 con 0x7f2c18108f90 2024-09-11T00:21:54.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.209+0000 7f2c17e00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2bfc0b5490 0x7f2bfc0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:54.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.209+0000 7f2c17e00640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2bfc0b5490 0x7f2bfc0b7950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c08005ab0 tx=0x7f2c08005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:54.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.210+0000 7f2c15600640 1 -- 172.21.15.103:0/4208921949 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2c100ad0d0 con 0x7f2c18108f90 2024-09-11T00:21:54.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.468+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2c181a5e60 con 0x7f2c18108f90 2024-09-11T00:21:54.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.473+0000 7f2c15600640 1 -- 172.21.15.103:0/4208921949 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+294224 (secure 0 0 0) 0x7f2c100ac9d0 con 0x7f2c18108f90 2024-09-11T00:21:54.479 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:54.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2bfc0b5490 msgr2=0x7f2bfc0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:54.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2bfc0b5490 0x7f2bfc0b7950 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c08005ab0 tx=0x7f2c08005a20 comp rx=0 tx=0).stop 2024-09-11T00:21:54.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c18108f90 msgr2=0x7f2c1819e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:54.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c18108f90 0x7f2c1819e950 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009fd0 tx=0x7f2c1000ef10 comp rx=0 tx=0).stop 2024-09-11T00:21:54.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 shutdown_connections 2024-09-11T00:21:54.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f2bfc0b5490 0x7f2bfc0b7950 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c18108f90 0x7f2c1819e950 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c18108610 0x7f2c181995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 --2- 172.21.15.103:0/4208921949 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c18102c00 0x7f2c18199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:54.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 >> 172.21.15.103:0/4208921949 conn(0x7f2c180fe6d0 msgr2=0x7f2c18107d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:54.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.477+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 shutdown_connections 2024-09-11T00:21:54.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:54.478+0000 7f2c1d800640 1 -- 172.21.15.103:0/4208921949 wait complete. 2024-09-11T00:21:54.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:54 smithi103 ceph-mon[30578]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-11T00:21:54.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:54 smithi183 ceph-mon[35934]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-11T00:21:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:54 smithi117 ceph-mon[35525]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-11T00:21:55.488 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":3157485146},{"type":"v1","addr":"172.21.15.103:6801","nonce":3157485146}]},"active_addr":"172.21.15.103:6801/3157485146","active_change":"2024-09-11T00:12:26.257459+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24181,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24194,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.103:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2830913090}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":4033999187}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":460259377}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":3484002684}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.103:0","nonce":2083013328}]}]} 2024-09-11T00:21:55.494 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T00:21:55.494 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T00:21:55.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:21:55.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:55 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4208921949' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:21:55.864 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:55.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:55 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4208921949' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:21:55.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:55 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4208921949' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-11T00:21:56.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.472+0000 7f71bfe00640 1 -- 172.21.15.103:0/2408790779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108610 msgr2=0x7f71c0108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:56.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.472+0000 7f71bfe00640 1 --2- 172.21.15.103:0/2408790779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108610 0x7f71c0108a50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009900 tx=0x7f71a802a410 comp rx=0 tx=0).stop 2024-09-11T00:21:56.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.473+0000 7f71bfe00640 1 -- 172.21.15.103:0/2408790779 shutdown_connections 2024-09-11T00:21:56.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.473+0000 7f71bfe00640 1 --2- 172.21.15.103:0/2408790779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c0108f90 0x7f71c010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.473+0000 7f71bfe00640 1 --2- 172.21.15.103:0/2408790779 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108610 0x7f71c0108a50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.473+0000 7f71bfe00640 1 --2- 172.21.15.103:0/2408790779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71c0102c00 0x7f71c0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.473+0000 7f71bfe00640 1 -- 172.21.15.103:0/2408790779 >> 172.21.15.103:0/2408790779 conn(0x7f71c00fe6d0 msgr2=0x7f71c0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.473+0000 7f71bfe00640 1 -- 172.21.15.103:0/2408790779 shutdown_connections 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.473+0000 7f71bfe00640 1 -- 172.21.15.103:0/2408790779 wait complete. 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 Processor -- start 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 -- start start 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71c0102c00 0x7f71c0076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:56.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c0108610 0x7f71c00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108f90 0x7f71c0075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0112c20 con 0x7f71c0108f90 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0112aa0 con 0x7f71c0102c00 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bfe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0112da0 con 0x7f71c0108610 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71b7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c0108610 0x7f71c00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bf400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108f90 0x7f71c0075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71bea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71c0102c00 0x7f71c0076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:56.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.474+0000 7f71b7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c0108610 0x7f71c00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57150/0 (socket says 172.21.15.103:57150) 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71b7e00640 1 -- 172.21.15.103:0/762488842 learned_addr learned my addr 172.21.15.103:0/762488842 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71bea00640 1 -- 172.21.15.103:0/762488842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c0108610 msgr2=0x7f71c00754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71bea00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c0108610 0x7f71c00754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71bea00640 1 -- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108f90 msgr2=0x7f71c0075ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71bea00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108f90 0x7f71c0075ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71bea00640 1 -- 172.21.15.103:0/762488842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c01ab9d0 con 0x7f71c0102c00 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71bea00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71c0102c00 0x7f71c0076f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71b000e9f0 tx=0x7f71b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:56.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71b7400640 1 -- 172.21.15.103:0/762488842 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b000ccc0 con 0x7f71c0102c00 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01abba0 con 0x7f71c0102c00 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.475+0000 7f71b7400640 1 -- 172.21.15.103:0/762488842 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71b00108e0 con 0x7f71c0102c00 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.476+0000 7f71b7400640 1 -- 172.21.15.103:0/762488842 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0018880 con 0x7f71c0102c00 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.476+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01ac100 con 0x7f71c0102c00 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.477+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c0102fe0 con 0x7f71c0102c00 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.479+0000 7f71b7400640 1 -- 172.21.15.103:0/762488842 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f71b0010430 con 0x7f71c0102c00 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.480+0000 7f71b7400640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71a40b5490 0x7f71a40b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.480+0000 7f71b7400640 1 -- 172.21.15.103:0/762488842 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f71b00e63d0 con 0x7f71c0102c00 2024-09-11T00:21:56.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.480+0000 7f71b7e00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71a40b5490 0x7f71a40b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:56.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.481+0000 7f71b7e00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71a40b5490 0x7f71a40b7950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009230 tx=0x7f71a80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:56.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.482+0000 7f71b7400640 1 -- 172.21.15.103:0/762488842 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f71b00ac720 con 0x7f71c0102c00 2024-09-11T00:21:56.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:56 smithi103 ceph-mon[30578]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-11T00:21:56.676 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.674+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71c002e5c0 con 0x7f71c0102c00 2024-09-11T00:21:56.678 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.677+0000 7f71b7400640 1 -- 172.21.15.103:0/762488842 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24772 (secure 0 0 0) 0x7f71b00a6e20 con 0x7f71c0102c00 2024-09-11T00:21:56.679 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:56.679 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":87,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:21:28.211213+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:21:56.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.678+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71a40b5490 msgr2=0x7f71a40b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:56.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.678+0000 7f71bfe00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71a40b5490 0x7f71a40b7950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f71a8009230 tx=0x7f71a80058d0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71c0102c00 msgr2=0x7f71c0076f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:56.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71c0102c00 0x7f71c0076f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f71b000e9f0 tx=0x7f71b000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 shutdown_connections 2024-09-11T00:21:56.681 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71a40b5490 0x7f71a40b7950 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71c0108f90 0x7f71c0075ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71c0108610 0x7f71c00754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 --2- 172.21.15.103:0/762488842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71c0102c00 0x7f71c0076f50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:56.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 >> 172.21.15.103:0/762488842 conn(0x7f71c00fe6d0 msgr2=0x7f71c0100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:56.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 shutdown_connections 2024-09-11T00:21:56.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:56.679+0000 7f71bfe00640 1 -- 172.21.15.103:0/762488842 wait complete. 2024-09-11T00:21:56.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:56 smithi183 ceph-mon[35934]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-11T00:21:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:56 smithi117 ceph-mon[35525]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-11T00:21:57.617 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T00:21:57.617 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:21:57.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:57 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/762488842' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:57.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:57 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/762488842' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:57 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/762488842' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:57.987 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:21:58.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 -- 172.21.15.103:0/3770491789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226410a800 msgr2=0x7f226410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:58.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 --2- 172.21.15.103:0/3770491789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226410a800 0x7f226410ac40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2250009900 tx=0x7f225002a410 comp rx=0 tx=0).stop 2024-09-11T00:21:58.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 -- 172.21.15.103:0/3770491789 shutdown_connections 2024-09-11T00:21:58.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 --2- 172.21.15.103:0/3770491789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226410b180 0x7f2264111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 --2- 172.21.15.103:0/3770491789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f226410a800 0x7f226410ac40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 --2- 172.21.15.103:0/3770491789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2264104df0 0x7f22641051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 -- 172.21.15.103:0/3770491789 >> 172.21.15.103:0/3770491789 conn(0x7f2264100840 msgr2=0x7f2264102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 -- 172.21.15.103:0/3770491789 shutdown_connections 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.607+0000 7f226ac00640 1 -- 172.21.15.103:0/3770491789 wait complete. 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.608+0000 7f226ac00640 1 Processor -- start 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226ac00640 1 -- start start 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226ac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264104df0 0x7f226419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226ac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226410a800 0x7f226419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226410b180 0x7f22641a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2264114dd0 con 0x7f226410b180 2024-09-11T00:21:58.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226ac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2264114c50 con 0x7f2264104df0 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226ac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2264114f50 con 0x7f226410a800 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226410b180 0x7f22641a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f2269800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264104df0 0x7f226419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226410b180 0x7f22641a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40682/0 (socket says 172.21.15.103:40682) 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f226a200640 1 -- 172.21.15.103:0/34852943 learned_addr learned my addr 172.21.15.103:0/34852943 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.609+0000 7f2268e00640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226410a800 0x7f226419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226a200640 1 -- 172.21.15.103:0/34852943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226410a800 msgr2=0x7f226419b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226a200640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226410a800 0x7f226419b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226a200640 1 -- 172.21.15.103:0/34852943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264104df0 msgr2=0x7f226419b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226a200640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264104df0 0x7f226419b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226a200640 1 -- 172.21.15.103:0/34852943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22641a1260 con 0x7f226410b180 2024-09-11T00:21:58.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f2269800640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264104df0 0x7f226419b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226a200640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226410b180 0x7f22641a0a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e970 tx=0x7f225c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f2262000640 1 -- 172.21.15.103:0/34852943 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c00ccc0 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f2262000640 1 -- 172.21.15.103:0/34852943 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f225c00ce60 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22641a1550 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f2262000640 1 -- 172.21.15.103:0/34852943 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c010430 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.610+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22641a7c10 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.611+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22641051d0 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.613+0000 7f2262000640 1 -- 172.21.15.103:0/34852943 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f225c004ff0 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.615+0000 7f2262000640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f224c0b5410 0x7f224c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.615+0000 7f2262000640 1 -- 172.21.15.103:0/34852943 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f225c0e6500 con 0x7f226410b180 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.615+0000 7f2269800640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f224c0b5410 0x7f224c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.616+0000 7f2269800640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f224c0b5410 0x7f224c0b78d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2258005fd0 tx=0x7f2258009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:21:58.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.619+0000 7f2262000640 1 -- 172.21.15.103:0/34852943 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f225c0acea0 con 0x7f226410b180 2024-09-11T00:21:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:58 smithi183 ceph-mon[35934]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-11T00:21:58.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.805+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f226410c9d0 con 0x7f226410b180 2024-09-11T00:21:58.807 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:58 smithi103 ceph-mon[30578]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-11T00:21:58.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.807+0000 7f2262000640 1 -- 172.21.15.103:0/34852943 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24772 (secure 0 0 0) 0x7f225c0ac7a0 con 0x7f226410b180 2024-09-11T00:21:58.808 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:21:58.808 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":87,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:21:28.211213+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f224c0b5410 msgr2=0x7f224c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f224c0b5410 0x7f224c0b78d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2258005fd0 tx=0x7f2258009500 comp rx=0 tx=0).stop 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226410b180 msgr2=0x7f22641a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226410b180 0x7f22641a0a90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f225c00e970 tx=0x7f225c00ed70 comp rx=0 tx=0).stop 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 shutdown_connections 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f224c0b5410 0x7f224c0b78d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f226410b180 0x7f22641a0a90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226410a800 0x7f226419b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 --2- 172.21.15.103:0/34852943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2264104df0 0x7f226419b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:21:58.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 >> 172.21.15.103:0/34852943 conn(0x7f2264100840 msgr2=0x7f2264102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:21:58.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 shutdown_connections 2024-09-11T00:21:58.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:21:58.810+0000 7f226ac00640 1 -- 172.21.15.103:0/34852943 wait complete. 2024-09-11T00:21:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:58 smithi117 ceph-mon[35525]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-11T00:21:59.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:21:59 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/34852943' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:59.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T00:21:59.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T00:21:59.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T00:21:59.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T00:21:59.761 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T00:21:59.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T00:21:59.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T00:21:59.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T00:21:59.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T00:21:59.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T00:21:59.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T00:21:59.764 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T00:21:59.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:21:59 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/34852943' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:21:59.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:21:59 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/34852943' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:22:00.716 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:00.760 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:00 smithi183 ceph-mon[35934]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:22:00.788 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:00.793 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:00.805 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:00.823 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:00.869 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:00 smithi103 ceph-mon[30578]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:22:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:00 smithi117 ceph-mon[35525]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:22:02.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 -- 172.21.15.103:0/1112151260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee810b330 msgr2=0x7feee8111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:02.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 --2- 172.21.15.103:0/1112151260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee810b330 0x7feee8111b30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feee0009ca0 tx=0x7feee002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 -- 172.21.15.103:0/1112151260 shutdown_connections 2024-09-11T00:22:02.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 --2- 172.21.15.103:0/1112151260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee810b330 0x7feee8111b30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 --2- 172.21.15.103:0/1112151260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee810a9b0 0x7feee810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 --2- 172.21.15.103:0/1112151260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee8104fa0 0x7feee8105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 -- 172.21.15.103:0/1112151260 >> 172.21.15.103:0/1112151260 conn(0x7feee81009f0 msgr2=0x7feee8102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 -- 172.21.15.103:0/1112151260 shutdown_connections 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.036+0000 7feeeca00640 1 -- 172.21.15.103:0/1112151260 wait complete. 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.037+0000 7feeeca00640 1 Processor -- start 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feeeca00640 1 -- start start 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feeeca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104fa0 0x7feee819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feeeca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee810a9b0 0x7feee819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feeeca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee810b330 0x7feee81a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feeeca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feee8114fb0 con 0x7feee810b330 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feeeca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feee8114e30 con 0x7feee810a9b0 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feeeca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feee8115130 con 0x7feee8104fa0 2024-09-11T00:22:02.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feee6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee810b330 0x7feee81a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feee5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104fa0 0x7feee819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feee5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104fa0 0x7feee819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:45520/0 (socket says 172.21.15.103:45520) 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.038+0000 7feee5e00640 1 -- 172.21.15.103:0/3405606159 learned_addr learned my addr 172.21.15.103:0/3405606159 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feee5400640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee810a9b0 0x7feee819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feee5e00640 1 -- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee810a9b0 msgr2=0x7feee819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feee5e00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feee810a9b0 0x7feee819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feee5e00640 1 -- 172.21.15.103:0/3405606159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee810b330 msgr2=0x7feee81a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feee5e00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feee810b330 0x7feee81a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feee5e00640 1 -- 172.21.15.103:0/3405606159 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee81a1510 con 0x7feee8104fa0 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feee5e00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104fa0 0x7feee819b460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feed800e9f0 tx=0x7feed800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed800cd80 con 0x7feee8104fa0 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed80108e0 con 0x7feee8104fa0 2024-09-11T00:22:02.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8018980 con 0x7feee8104fa0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.039+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee81a1800 con 0x7feee8104fa0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.040+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee81a7ec0 con 0x7feee8104fa0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.041+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feee80638e0 con 0x7feee8104fa0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.043+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7feed8010430 con 0x7feee8104fa0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.044+0000 7feedea00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feecc0b53f0 0x7feecc0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.045+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7feed80e6b40 con 0x7feee8104fa0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.045+0000 7feee5400640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feecc0b53f0 0x7feecc0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.045+0000 7feedea00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] conn(0x7feecc0c04b0 0x7feecc0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.045+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 --> [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feed801bec0 con 0x7feecc0c04b0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.045+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7feed80e6ef0 con 0x7feee8104fa0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.045+0000 7feee6800640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] conn(0x7feecc0c04b0 0x7feecc0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.046+0000 7feee5400640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feecc0b53f0 0x7feecc0b78b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feee8109450 tx=0x7feed4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.046+0000 7feee6800640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] conn(0x7feecc0c04b0 0x7feecc0c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:02.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.047+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== osd.4 v2:172.21.15.117:6808/392780746 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feed801bec0 con 0x7feecc0c04b0 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.061+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 --> [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feee8063db0 con 0x7feecc0c04b0 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feedea00640 1 -- 172.21.15.103:0/3405606159 <== osd.4 v2:172.21.15.117:6808/392780746 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feee8063db0 con 0x7feecc0c04b0 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] conn(0x7feecc0c04b0 msgr2=0x7feecc0c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] conn(0x7feecc0c04b0 0x7feecc0c2950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feecc0b53f0 msgr2=0x7feecc0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feecc0b53f0 0x7feecc0b78b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7feee8109450 tx=0x7feed4005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104fa0 msgr2=0x7feee819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 --2- 172.21.15.103:0/3405606159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feee8104fa0 0x7feee819b460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feed800e9f0 tx=0x7feed800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feee6800640 1 -- 172.21.15.103:0/3405606159 reap_dead start 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 shutdown_connections 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 >> 172.21.15.103:0/3405606159 conn(0x7feee81009f0 msgr2=0x7feee8102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.062+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 shutdown_connections 2024-09-11T00:22:02.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:02.063+0000 7feeeca00640 1 -- 172.21.15.103:0/3405606159 wait complete. 2024-09-11T00:22:02.084 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:02 smithi183 ceph-mon[35934]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-11T00:22:02.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:02 smithi103 ceph-mon[30578]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-11T00:22:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:02 smithi117 ceph-mon[35525]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-11T00:22:03.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 -- 172.21.15.103:0/2389066942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108fa0 msgr2=0x7f3ab410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 --2- 172.21.15.103:0/2389066942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108fa0 0x7f3ab410f7a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009900 tx=0x7f3aac02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.050 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 -- 172.21.15.103:0/2389066942 shutdown_connections 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 --2- 172.21.15.103:0/2389066942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108fa0 0x7f3ab410f7a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 --2- 172.21.15.103:0/2389066942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab4108620 0x7f3ab4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 --2- 172.21.15.103:0/2389066942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ab4102c10 0x7f3ab4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 -- 172.21.15.103:0/2389066942 >> 172.21.15.103:0/2389066942 conn(0x7f3ab40fe660 msgr2=0x7f3ab4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 -- 172.21.15.103:0/2389066942 shutdown_connections 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.028+0000 7f3abac00640 1 -- 172.21.15.103:0/2389066942 wait complete. 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.029+0000 7f3abac00640 1 Processor -- start 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.029+0000 7f3abac00640 1 -- start start 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3abac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab4102c10 0x7f3ab419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3abac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108620 0x7f3ab419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3abac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ab4108fa0 0x7f3ab41a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3abac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4112c60 con 0x7f3ab4108620 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3abac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4112ae0 con 0x7f3ab4102c10 2024-09-11T00:22:03.051 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3abac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4112de0 con 0x7f3ab4108fa0 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108620 0x7f3ab419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3aba200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ab4108fa0 0x7f3ab41a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3ab9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab4102c10 0x7f3ab419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3ab8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108620 0x7f3ab419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57112/0 (socket says 172.21.15.103:57112) 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3ab8e00640 1 -- 172.21.15.103:0/405008532 learned_addr learned my addr 172.21.15.103:0/405008532 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.030+0000 7f3aba200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ab4108fa0 0x7f3ab41a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:45544/0 (socket says 172.21.15.103:45544) 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3aba200640 1 -- 172.21.15.103:0/405008532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab4102c10 msgr2=0x7f3ab419b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3aba200640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ab4102c10 0x7f3ab419b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3aba200640 1 -- 172.21.15.103:0/405008532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108620 msgr2=0x7f3ab419b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3aba200640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ab4108620 0x7f3ab419b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3aba200640 1 -- 172.21.15.103:0/405008532 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab41a1280 con 0x7f3ab4108fa0 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3aba200640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ab4108fa0 0x7f3ab41a0ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005940 tx=0x7f3aac031a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:03.052 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac03f030 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac0043d0 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac035070 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab41a1510 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.031+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a7c30 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.032+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3ab4075da0 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.035+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f3aac003c30 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.036+0000 7f3ab2000640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3a980b5410 0x7f3a980b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.037+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f3aac106c80 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.037+0000 7f3ab9800640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3a980b5410 0x7f3a980b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.037+0000 7f3ab2000640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] conn(0x7f3a980c04d0 0x7f3a980c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.037+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 --> [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a980c3040 con 0x7f3a980c04d0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.037+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f3aac107030 con 0x7f3ab4108fa0 2024-09-11T00:22:03.053 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.037+0000 7f3ab9800640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3a980b5410 0x7f3a980b78d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:03.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.037+0000 7f3ab8e00640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] conn(0x7f3a980c04d0 0x7f3a980c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.038+0000 7f3ab8e00640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] conn(0x7f3a980c04d0 0x7f3a980c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:03.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.039+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== osd.1 v2:172.21.15.117:6800/2016738362 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a980c3040 con 0x7f3a980c04d0 2024-09-11T00:22:03.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.053+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 --> [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3ab4103ec0 con 0x7f3a980c04d0 2024-09-11T00:22:03.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.053+0000 7f3ab2000640 1 -- 172.21.15.103:0/405008532 <== osd.1 v2:172.21.15.117:6800/2016738362 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3ab4103ec0 con 0x7f3a980c04d0 2024-09-11T00:22:03.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.054+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 >> [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] conn(0x7f3a980c04d0 msgr2=0x7f3a980c2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.054+0000 7f3abac00640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] conn(0x7f3a980c04d0 0x7f3a980c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.054+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3a980b5410 msgr2=0x7f3a980b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.054+0000 7f3abac00640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3a980b5410 0x7f3a980b78d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4009500 comp rx=0 tx=0).stop 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.054+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ab4108fa0 msgr2=0x7f3ab41a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.054+0000 7f3abac00640 1 --2- 172.21.15.103:0/405008532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3ab4108fa0 0x7f3ab41a0ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005940 tx=0x7f3aac031a10 comp rx=0 tx=0).stop 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.054+0000 7f3aba200640 1 -- 172.21.15.103:0/405008532 reap_dead start 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.055+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 shutdown_connections 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.055+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 >> 172.21.15.103:0/405008532 conn(0x7f3ab40fe660 msgr2=0x7f3ab4075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.055+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 shutdown_connections 2024-09-11T00:22:03.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.055+0000 7f3abac00640 1 -- 172.21.15.103:0/405008532 wait complete. 2024-09-11T00:22:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 -- 172.21.15.103:0/3130030831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c104e10 msgr2=0x7f616c1051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 --2- 172.21.15.103:0/3130030831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c104e10 0x7f616c1051f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6160009900 tx=0x7f616002a410 comp rx=0 tx=0).stop 2024-09-11T00:22:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 -- 172.21.15.103:0/3130030831 shutdown_connections 2024-09-11T00:22:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 --2- 172.21.15.103:0/3130030831 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f616c10b1a0 0x7f616c1119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 --2- 172.21.15.103:0/3130030831 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616c10a820 0x7f616c10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 --2- 172.21.15.103:0/3130030831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c104e10 0x7f616c1051f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 -- 172.21.15.103:0/3130030831 >> 172.21.15.103:0/3130030831 conn(0x7f616c100840 msgr2=0x7f616c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 -- 172.21.15.103:0/3130030831 shutdown_connections 2024-09-11T00:22:03.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.723+0000 7f6173000640 1 -- 172.21.15.103:0/3130030831 wait complete. 2024-09-11T00:22:03.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.724+0000 7f6173000640 1 Processor -- start 2024-09-11T00:22:03.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.724+0000 7f6173000640 1 -- start start 2024-09-11T00:22:03.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6173000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616c104e10 0x7f616c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6173000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f616c10a820 0x7f616c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6173000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c10b1a0 0x7f616c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6173000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f616c114e60 con 0x7f616c10b1a0 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6173000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f616c114ce0 con 0x7f616c104e10 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6173000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f616c114fe0 con 0x7f616c10a820 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6172600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c10b1a0 0x7f616c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616c104e10 0x7f616c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6172600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c10b1a0 0x7f616c1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57136/0 (socket says 172.21.15.103:57136) 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6172600640 1 -- 172.21.15.103:0/2205319587 learned_addr learned my addr 172.21.15.103:0/2205319587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171c00640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616c104e10 0x7f616c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:34868/0 (socket says 172.21.15.103:34868) 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171200640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f616c10a820 0x7f616c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171c00640 1 -- 172.21.15.103:0/2205319587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f616c10a820 msgr2=0x7f616c19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171c00640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f616c10a820 0x7f616c19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171c00640 1 -- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c10b1a0 msgr2=0x7f616c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171c00640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c10b1a0 0x7f616c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6171c00640 1 -- 172.21.15.103:0/2205319587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c1a13c0 con 0x7f616c104e10 2024-09-11T00:22:03.729 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.725+0000 7f6172600640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f616c10b1a0 0x7f616c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:03.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.726+0000 7f6171c00640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616c104e10 0x7f616c19b310 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f616002a930 tx=0x7f6160031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:03.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.726+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616003f030 con 0x7f616c104e10 2024-09-11T00:22:03.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.726+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6160004490 con 0x7f616c104e10 2024-09-11T00:22:03.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.726+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c1a1650 con 0x7f616c104e10 2024-09-11T00:22:03.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.726+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6160035070 con 0x7f616c104e10 2024-09-11T00:22:03.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.726+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c1a7d70 con 0x7f616c104e10 2024-09-11T00:22:03.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.727+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f616c0638a0 con 0x7f616c104e10 2024-09-11T00:22:03.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.730+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f616003ba30 con 0x7f616c104e10 2024-09-11T00:22:03.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.731+0000 7f616a000640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f61540b5490 0x7f61540b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.731+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f6160106c90 con 0x7f616c104e10 2024-09-11T00:22:03.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.731+0000 7f6171200640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f61540b5490 0x7f61540b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.732+0000 7f616a000640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] conn(0x7f61540c0550 0x7f61540c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:03.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.732+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 --> [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6160032970 con 0x7f61540c0550 2024-09-11T00:22:03.733 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.732+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f6160107040 con 0x7f616c104e10 2024-09-11T00:22:03.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.732+0000 7f6172600640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] conn(0x7f61540c0550 0x7f61540c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:03.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.732+0000 7f6171200640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f61540b5490 0x7f61540b7950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f616c103e50 tx=0x7f6158005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:03.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.732+0000 7f6172600640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] conn(0x7f61540c0550 0x7f61540c29f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:03.735 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.733+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== osd.3 v2:172.21.15.103:6810/2768325174 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6160032970 con 0x7f61540c0550 2024-09-11T00:22:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.747+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 --> [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f616c063cd0 con 0x7f61540c0550 2024-09-11T00:22:03.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f616a000640 1 -- 172.21.15.103:0/2205319587 <== osd.3 v2:172.21.15.103:6810/2768325174 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f616c063cd0 con 0x7f61540c0550 2024-09-11T00:22:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] conn(0x7f61540c0550 msgr2=0x7f61540c29f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] conn(0x7f61540c0550 0x7f61540c29f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f61540b5490 msgr2=0x7f61540b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f61540b5490 0x7f61540b7950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f616c103e50 tx=0x7f6158005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616c104e10 msgr2=0x7f616c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 --2- 172.21.15.103:0/2205319587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616c104e10 0x7f616c19b310 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f616002a930 tx=0x7f6160031a20 comp rx=0 tx=0).stop 2024-09-11T00:22:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6172600640 1 -- 172.21.15.103:0/2205319587 reap_dead start 2024-09-11T00:22:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 shutdown_connections 2024-09-11T00:22:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.748+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 >> 172.21.15.103:0/2205319587 conn(0x7f616c100840 msgr2=0x7f616c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.749+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 shutdown_connections 2024-09-11T00:22:03.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.749+0000 7f6173000640 1 -- 172.21.15.103:0/2205319587 wait complete. 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 -- 172.21.15.103:0/627744605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bc8104fa0 msgr2=0x7f9bc8105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 --2- 172.21.15.103:0/627744605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bc8104fa0 0x7f9bc8105380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009860 tx=0x7f9bb802a340 comp rx=0 tx=0).stop 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 -- 172.21.15.103:0/627744605 shutdown_connections 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 --2- 172.21.15.103:0/627744605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bc810b330 0x7f9bc8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 --2- 172.21.15.103:0/627744605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc810a9b0 0x7f9bc810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 --2- 172.21.15.103:0/627744605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bc8104fa0 0x7f9bc8105380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 -- 172.21.15.103:0/627744605 >> 172.21.15.103:0/627744605 conn(0x7f9bc8100a50 msgr2=0x7f9bc8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 -- 172.21.15.103:0/627744605 shutdown_connections 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.609+0000 7f9bcde00640 1 -- 172.21.15.103:0/627744605 wait complete. 2024-09-11T00:22:04.176 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.610+0000 7f9bcde00640 1 Processor -- start 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bcde00640 1 -- start start 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bcde00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bc8104fa0 0x7f9bc819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bcde00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc810a9b0 0x7f9bc819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bcde00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bc810b330 0x7f9bc81a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bcde00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8114f60 con 0x7f9bc810b330 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bcde00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8114de0 con 0x7f9bc8104fa0 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bcde00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc81150e0 con 0x7f9bc810a9b0 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bc6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc810a9b0 0x7f9bc819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bc6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bc8104fa0 0x7f9bc819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bc7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bc810b330 0x7f9bc81a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bc6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc810a9b0 0x7f9bc819ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:45568/0 (socket says 172.21.15.103:45568) 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.611+0000 7f9bc6000640 1 -- 172.21.15.103:0/489586787 learned_addr learned my addr 172.21.15.103:0/489586787 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:04.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bc6000640 1 -- 172.21.15.103:0/489586787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bc8104fa0 msgr2=0x7f9bc819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bc6000640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9bc8104fa0 0x7f9bc819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bc6000640 1 -- 172.21.15.103:0/489586787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bc810b330 msgr2=0x7f9bc81a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bc6000640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9bc810b330 0x7f9bc81a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bc6000640 1 -- 172.21.15.103:0/489586787 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc81a15e0 con 0x7f9bc810a9b0 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bc6000640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc810a9b0 0x7f9bc819ba70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400e9f0 tx=0x7f9bb400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb400cd80 con 0x7f9bc810a9b0 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc81a18d0 con 0x7f9bc810a9b0 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb40054d0 con 0x7f9bc810a9b0 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4010690 con 0x7f9bc810a9b0 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.612+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc81a7f90 con 0x7f9bc810a9b0 2024-09-11T00:22:04.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.613+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9bc80638e0 con 0x7f9bc810a9b0 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.616+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f9bb4004e30 con 0x7f9bc810a9b0 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.617+0000 7f9bcca00640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9bac0b5370 0x7f9bac0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.617+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9bb40e6940 con 0x7f9bc810a9b0 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.618+0000 7f9bc6a00640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9bac0b5370 0x7f9bac0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.618+0000 7f9bcca00640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] conn(0x7f9bac0c0430 0x7f9bac0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.618+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 --> [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bb401ee50 con 0x7f9bac0c0430 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.618+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9bb40e6cf0 con 0x7f9bc810a9b0 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.618+0000 7f9bc7400640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] conn(0x7f9bac0c0430 0x7f9bac0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.618+0000 7f9bc6a00640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9bac0b5370 0x7f9bac0b7830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009830 tx=0x7f9bb8030340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.619+0000 7f9bc7400640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] conn(0x7f9bac0c0430 0x7f9bac0c28d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.620+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== osd.2 v2:172.21.15.183:6800/4126324003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bb401ee50 con 0x7f9bac0c0430 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.648+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 --> [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9bc8063d60 con 0x7f9bac0c0430 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.649+0000 7f9bcca00640 1 -- 172.21.15.103:0/489586787 <== osd.2 v2:172.21.15.183:6800/4126324003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9bc8063d60 con 0x7f9bac0c0430 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.649+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] conn(0x7f9bac0c0430 msgr2=0x7f9bac0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.649+0000 7f9bcde00640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] conn(0x7f9bac0c0430 0x7f9bac0c28d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9bac0b5370 msgr2=0x7f9bac0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9bac0b5370 0x7f9bac0b7830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009830 tx=0x7f9bb8030340 comp rx=0 tx=0).stop 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc810a9b0 msgr2=0x7f9bc819ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 --2- 172.21.15.103:0/489586787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9bc810a9b0 0x7f9bc819ba70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400e9f0 tx=0x7f9bb400edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bc7400640 1 -- 172.21.15.103:0/489586787 reap_dead start 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 shutdown_connections 2024-09-11T00:22:04.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 >> 172.21.15.103:0/489586787 conn(0x7f9bc8100a50 msgr2=0x7f9bc8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:04.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 shutdown_connections 2024-09-11T00:22:04.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:03.650+0000 7f9bcde00640 1 -- 172.21.15.103:0/489586787 wait complete. 2024-09-11T00:22:04.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:04 smithi103 ceph-mon[30578]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-11T00:22:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:04 smithi183 ceph-mon[35934]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-11T00:22:04.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 -- 172.21.15.103:0/908374073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04600754c0 msgr2=0x7f04600758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 --2- 172.21.15.103:0/908374073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04600754c0 0x7f04600758a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0450009900 tx=0x7f045002a410 comp rx=0 tx=0).stop 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 -- 172.21.15.103:0/908374073 shutdown_connections 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 --2- 172.21.15.103:0/908374073 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f046010f220 0x7f0460111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 --2- 172.21.15.103:0/908374073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0460075de0 0x7f046010ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 --2- 172.21.15.103:0/908374073 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f04600754c0 0x7f04600758a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 -- 172.21.15.103:0/908374073 >> 172.21.15.103:0/908374073 conn(0x7f04600fe660 msgr2=0x7f0460100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 -- 172.21.15.103:0/908374073 shutdown_connections 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.505+0000 7f0464a00640 1 -- 172.21.15.103:0/908374073 wait complete. 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.506+0000 7f0464a00640 1 Processor -- start 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.506+0000 7f0464a00640 1 -- start start 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f0464a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04600754c0 0x7f046019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f0464a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0460075de0 0x7f046019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f0464a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f046010f220 0x7f04601a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f0464a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0460114e00 con 0x7f0460075de0 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f0464a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0460114c80 con 0x7f046010f220 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f0464a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0460114f80 con 0x7f04600754c0 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0460075de0 0x7f046019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04600754c0 0x7f046019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04600754c0 0x7f046019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:45604/0 (socket says 172.21.15.103:45604) 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045f400640 1 -- 172.21.15.103:0/3273420329 learned_addr learned my addr 172.21.15.103:0/3273420329 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045fe00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f046010f220 0x7f04601a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045fe00640 1 -- 172.21.15.103:0/3273420329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04600754c0 msgr2=0x7f046019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045fe00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f04600754c0 0x7f046019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045fe00640 1 -- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0460075de0 msgr2=0x7f046019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045fe00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0460075de0 0x7f046019b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.507+0000 7f045fe00640 1 -- 172.21.15.103:0/3273420329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04601a1300 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.508+0000 7f045fe00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f046010f220 0x7f04601a0b30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f046006bf90 tx=0x7f045800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.508+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045800cdf0 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.508+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0458005020 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.508+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458010690 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.508+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04601a15f0 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.508+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04601a7cb0 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.509+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04601005c0 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.511+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f0458005460 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.513+0000 7f0457e00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f04440b5410 0x7f04440b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.513+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f04580e6950 con 0x7f046010f220 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.513+0000 7f045f400640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f04440b5410 0x7f04440b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.513+0000 7f0457e00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] conn(0x7f04440c04d0 0x7f04440c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:04.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.513+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 --> [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0458016e50 con 0x7f04440c04d0 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.513+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f04580e6d00 con 0x7f046010f220 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.513+0000 7f045ea00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] conn(0x7f04440c04d0 0x7f04440c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.514+0000 7f045f400640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f04440b5410 0x7f04440b78d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0450031440 tx=0x7f04500058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.514+0000 7f045ea00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] conn(0x7f04440c04d0 0x7f04440c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.515+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== osd.6 v2:172.21.15.103:6818/962000049 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0458016e50 con 0x7f04440c04d0 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.529+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 --> [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04600769d0 con 0x7f04440c04d0 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.529+0000 7f0457e00640 1 -- 172.21.15.103:0/3273420329 <== osd.6 v2:172.21.15.103:6818/962000049 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f04600769d0 con 0x7f04440c04d0 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] conn(0x7f04440c04d0 msgr2=0x7f04440c2970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] conn(0x7f04440c04d0 0x7f04440c2970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f04440b5410 msgr2=0x7f04440b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f04440b5410 0x7f04440b78d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0450031440 tx=0x7f04500058d0 comp rx=0 tx=0).stop 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f046010f220 msgr2=0x7f04601a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 --2- 172.21.15.103:0/3273420329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f046010f220 0x7f04601a0b30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f046006bf90 tx=0x7f045800ef10 comp rx=0 tx=0).stop 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f045fe00640 1 -- 172.21.15.103:0/3273420329 reap_dead start 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 shutdown_connections 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 >> 172.21.15.103:0/3273420329 conn(0x7f04600fe660 msgr2=0x7f04601113b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 shutdown_connections 2024-09-11T00:22:04.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:04.530+0000 7f0464a00640 1 -- 172.21.15.103:0/3273420329 wait complete. 2024-09-11T00:22:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:04 smithi117 ceph-mon[35525]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 -- 172.21.15.103:0/3146378750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24101850 msgr2=0x7fbb24101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 --2- 172.21.15.103:0/3146378750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24101850 0x7fbb24101c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009900 tx=0x7fbb1802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 -- 172.21.15.103:0/3146378750 shutdown_connections 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 --2- 172.21.15.103:0/3146378750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb2410f400 0x7fbb241117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 --2- 172.21.15.103:0/3146378750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb24102170 0x7fbb2410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 --2- 172.21.15.103:0/3146378750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb24101850 0x7fbb24101c30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 -- 172.21.15.103:0/3146378750 >> 172.21.15.103:0/3146378750 conn(0x7fbb240fd700 msgr2=0x7fbb240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 -- 172.21.15.103:0/3146378750 shutdown_connections 2024-09-11T00:22:06.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.232+0000 7fbb2c800640 1 -- 172.21.15.103:0/3146378750 wait complete. 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.233+0000 7fbb2c800640 1 Processor -- start 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.233+0000 7fbb2c800640 1 -- start start 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2c800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb24101850 0x7fbb2419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2c800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24102170 0x7fbb2419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2c800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb2410f400 0x7fbb241a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2c800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24114fb0 con 0x7fbb2410f400 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2c800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24114e30 con 0x7fbb24102170 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2c800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb24115130 con 0x7fbb24101850 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb2410f400 0x7fbb241a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb2410f400 0x7fbb241a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57182/0 (socket says 172.21.15.103:57182) 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2a800640 1 -- 172.21.15.103:0/1353226649 learned_addr learned my addr 172.21.15.103:0/1353226649 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb29e00640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb24101850 0x7fbb2419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb29400640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24102170 0x7fbb2419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.234+0000 7fbb2a800640 1 -- 172.21.15.103:0/1353226649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb24101850 msgr2=0x7fbb2419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb2a800640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb24101850 0x7fbb2419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb2a800640 1 -- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24102170 msgr2=0x7fbb2419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb2a800640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb24102170 0x7fbb2419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb2a800640 1 -- 172.21.15.103:0/1353226649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb241a1570 con 0x7fbb2410f400 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb2a800640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb2410f400 0x7fbb241a0da0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000eb40 tx=0x7fbb2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20004b20 con 0x7fbb2410f400 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb20004cc0 con 0x7fbb2410f400 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb241a1860 con 0x7fbb2410f400 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20010720 con 0x7fbb2410f400 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.235+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb241a7f20 con 0x7fbb2410f400 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.236+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb24110ac0 con 0x7fbb2410f400 2024-09-11T00:22:06.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.238+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fbb20005560 con 0x7fbb2410f400 2024-09-11T00:22:06.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.240+0000 7fbb1ea00640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbb0c0b5370 0x7fbb0c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.240+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fbb200e6a10 con 0x7fbb2410f400 2024-09-11T00:22:06.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.240+0000 7fbb29e00640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbb0c0b5370 0x7fbb0c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.240+0000 7fbb1ea00640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] conn(0x7fbb0c0c0430 0x7fbb0c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:06.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.240+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 --> [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb0c0c2fa0 con 0x7fbb0c0c0430 2024-09-11T00:22:06.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.240+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fbb200e6dc0 con 0x7fbb2410f400 2024-09-11T00:22:06.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.241+0000 7fbb29400640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] conn(0x7fbb0c0c0430 0x7fbb0c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:06.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.241+0000 7fbb29e00640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbb0c0b5370 0x7fbb0c0b7830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005ab0 tx=0x7fbb18005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.241+0000 7fbb29400640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] conn(0x7fbb0c0c0430 0x7fbb0c0c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:06.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.242+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== osd.7 v2:172.21.15.117:6816/2935555111 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb0c0c2fa0 con 0x7fbb0c0c0430 2024-09-11T00:22:06.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.256+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 --> [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb2406bc10 con 0x7fbb0c0c0430 2024-09-11T00:22:06.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.257+0000 7fbb1ea00640 1 -- 172.21.15.103:0/1353226649 <== osd.7 v2:172.21.15.117:6816/2935555111 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb2406bc10 con 0x7fbb0c0c0430 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] conn(0x7fbb0c0c0430 msgr2=0x7fbb0c0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] conn(0x7fbb0c0c0430 0x7fbb0c0c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbb0c0b5370 msgr2=0x7fbb0c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbb0c0b5370 0x7fbb0c0b7830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005ab0 tx=0x7fbb18005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb2410f400 msgr2=0x7fbb241a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 --2- 172.21.15.103:0/1353226649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb2410f400 0x7fbb241a0da0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000eb40 tx=0x7fbb2000c2d0 comp rx=0 tx=0).stop 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2a800640 1 -- 172.21.15.103:0/1353226649 reap_dead start 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 shutdown_connections 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 >> 172.21.15.103:0/1353226649 conn(0x7fbb240fd700 msgr2=0x7fbb240ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 shutdown_connections 2024-09-11T00:22:06.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.258+0000 7fbb2c800640 1 -- 172.21.15.103:0/1353226649 wait complete. 2024-09-11T00:22:06.650 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:06 smithi103 ceph-mon[30578]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:06.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:06 smithi183 ceph-mon[35934]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:06.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:06 smithi117 ceph-mon[35525]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:07.192 INFO:teuthology.orchestra.run.smithi103.stdout:55834574935 2024-09-11T00:22:07.193 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T00:22:07.660 INFO:teuthology.orchestra.run.smithi103.stdout:133143986248 2024-09-11T00:22:07.660 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T00:22:08.182 INFO:teuthology.orchestra.run.smithi103.stdout:201863462967 2024-09-11T00:22:08.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T00:22:08.247 INFO:teuthology.orchestra.run.smithi103.stdout:107374182478 2024-09-11T00:22:08.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T00:22:08.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 -- 172.21.15.103:0/1177529309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 msgr2=0x7f53440758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 --2- 172.21.15.103:0/1177529309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 0x7f53440758c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5338009860 tx=0x7f533802a340 comp rx=0 tx=0).stop 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 -- 172.21.15.103:0/1177529309 shutdown_connections 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 --2- 172.21.15.103:0/1177529309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f534410f200 0x7f53441115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 --2- 172.21.15.103:0/1177529309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5344075e00 0x7f534410ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 --2- 172.21.15.103:0/1177529309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 0x7f53440758c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 -- 172.21.15.103:0/1177529309 >> 172.21.15.103:0/1177529309 conn(0x7f53440fe660 msgr2=0x7f5344100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 -- 172.21.15.103:0/1177529309 shutdown_connections 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.879+0000 7f534a000640 1 -- 172.21.15.103:0/1177529309 wait complete. 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.880+0000 7f534a000640 1 Processor -- start 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.880+0000 7f534a000640 1 -- start start 2024-09-11T00:22:08.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f534a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 0x7f534419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f534a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5344075e00 0x7f534419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f534a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f534410f200 0x7f53441a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f534a000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5344114da0 con 0x7f5344075e00 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f534a000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5344114c20 con 0x7f534410f200 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f534a000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5344114f20 con 0x7f53440754e0 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 0x7f534419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 0x7f534419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:45624/0 (socket says 172.21.15.103:45624) 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5343e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5344075e00 0x7f534419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 -- 172.21.15.103:0/3504269379 learned_addr learned my addr 172.21.15.103:0/3504269379 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5349600640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f534410f200 0x7f53441a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:08.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 -- 172.21.15.103:0/3504269379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f534410f200 msgr2=0x7f53441a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f534410f200 0x7f53441a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 -- 172.21.15.103:0/3504269379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5344075e00 msgr2=0x7f534419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5344075e00 0x7f534419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5348c00640 1 -- 172.21.15.103:0/3504269379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53441a1360 con 0x7f53440754e0 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.881+0000 7f5349600640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f534410f200 0x7f53441a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.882+0000 7f5348c00640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 0x7f534419b2b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5338035040 tx=0x7f5338032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.882+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338037040 con 0x7f53440754e0 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.882+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5338041070 con 0x7f53440754e0 2024-09-11T00:22:08.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.882+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338003850 con 0x7f53440754e0 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.882+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53441a15f0 con 0x7f53440754e0 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.882+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53441a7d10 con 0x7f53440754e0 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.883+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53441005c0 con 0x7f53440754e0 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.886+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f533804e070 con 0x7f53440754e0 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.888+0000 7f5341600640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f532c0b5490 0x7f532c0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.888+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f5338106590 con 0x7f53440754e0 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.888+0000 7f5343e00640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f532c0b5490 0x7f532c0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.888+0000 7f5341600640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] conn(0x7f532c0c0550 0x7f532c0c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:08.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.888+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 --> [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f533804be50 con 0x7f532c0c0550 2024-09-11T00:22:08.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.888+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f5338106940 con 0x7f53440754e0 2024-09-11T00:22:08.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.889+0000 7f5349600640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] conn(0x7f532c0c0550 0x7f532c0c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:08.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.889+0000 7f5343e00640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f532c0b5490 0x7f532c0b7950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5344100e00 tx=0x7f5330009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:08.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.889+0000 7f5349600640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] conn(0x7f532c0c0550 0x7f532c0c29f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:08.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.890+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== osd.11 v2:172.21.15.183:6824/405835574 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f533804be50 con 0x7f532c0c0550 2024-09-11T00:22:08.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.904+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 --> [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53440769f0 con 0x7f532c0c0550 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f5341600640 1 -- 172.21.15.103:0/3504269379 <== osd.11 v2:172.21.15.183:6824/405835574 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f53440769f0 con 0x7f532c0c0550 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] conn(0x7f532c0c0550 msgr2=0x7f532c0c29f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] conn(0x7f532c0c0550 0x7f532c0c29f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f532c0b5490 msgr2=0x7f532c0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f532c0b5490 0x7f532c0b7950 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5344100e00 tx=0x7f5330009260 comp rx=0 tx=0).stop 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 msgr2=0x7f534419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 --2- 172.21.15.103:0/3504269379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f53440754e0 0x7f534419b2b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5338035040 tx=0x7f5338032d50 comp rx=0 tx=0).stop 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f5349600640 1 -- 172.21.15.103:0/3504269379 reap_dead start 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 shutdown_connections 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.905+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 >> 172.21.15.103:0/3504269379 conn(0x7f53440fe660 msgr2=0x7f5344111370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.906+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 shutdown_connections 2024-09-11T00:22:08.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:06.906+0000 7f534a000640 1 -- 172.21.15.103:0/3504269379 wait complete. 2024-09-11T00:22:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:08 smithi183 ceph-mon[35934]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:08.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:08 smithi103 ceph-mon[30578]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:08 smithi117 ceph-mon[35525]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:09.498 INFO:teuthology.orchestra.run.smithi103.stdout:77309411411 2024-09-11T00:22:09.498 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T00:22:09.557 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:09.558 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:09.566 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:09.572 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:09.578 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:10.133 INFO:teuthology.orchestra.run.smithi103.stdout:180388626492 2024-09-11T00:22:10.134 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T00:22:10.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:10 smithi103 ceph-mon[30578]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:10.767 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:10.769 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:10 smithi183 ceph-mon[35934]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:10.802 INFO:teuthology.orchestra.run.smithi103.stdout:300647710754 2024-09-11T00:22:10.802 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T00:22:10.814 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:10 smithi117 ceph-mon[35525]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:11.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 -- 172.21.15.103:0/3024158850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28102c10 msgr2=0x7efc28102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:11.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 --2- 172.21.15.103:0/3024158850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28102c10 0x7efc28102ff0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efc18009900 tx=0x7efc1802a410 comp rx=0 tx=0).stop 2024-09-11T00:22:11.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 -- 172.21.15.103:0/3024158850 shutdown_connections 2024-09-11T00:22:11.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 --2- 172.21.15.103:0/3024158850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc28108fa0 0x7efc2810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:11.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 --2- 172.21.15.103:0/3024158850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc28108620 0x7efc28108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:11.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 --2- 172.21.15.103:0/3024158850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28102c10 0x7efc28102ff0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 -- 172.21.15.103:0/3024158850 >> 172.21.15.103:0/3024158850 conn(0x7efc280fe660 msgr2=0x7efc28100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 -- 172.21.15.103:0/3024158850 shutdown_connections 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.840+0000 7efc2d800640 1 -- 172.21.15.103:0/3024158850 wait complete. 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 Processor -- start 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 -- start start 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc28102c10 0x7efc2819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28108620 0x7efc2819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc28108fa0 0x7efc281a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc28112cb0 con 0x7efc28108620 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efc28112b30 con 0x7efc28102c10 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.841+0000 7efc2d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc28112e30 con 0x7efc28108fa0 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc28102c10 0x7efc2819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:11.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28108620 0x7efc2819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc28102c10 0x7efc2819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:34924/0 (socket says 172.21.15.103:34924) 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 -- 172.21.15.103:0/3628343776 learned_addr learned my addr 172.21.15.103:0/3628343776 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc2ce00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc28108fa0 0x7efc281a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 -- 172.21.15.103:0/3628343776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc28108fa0 msgr2=0x7efc281a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc28108fa0 0x7efc281a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 -- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28108620 msgr2=0x7efc2819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28108620 0x7efc2819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 -- 172.21.15.103:0/3628343776 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc281a1440 con 0x7efc28102c10 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27400640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc28108620 0x7efc2819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc2ce00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc28108fa0 0x7efc281a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.842+0000 7efc27e00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc28102c10 0x7efc2819b390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc18005ab0 tx=0x7efc18003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:11.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.843+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1803f070 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.843+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc281a16d0 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.843+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc18031b60 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.843+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18035030 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.843+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc281a7df0 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.844+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc280637e0 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.846+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7efc1803b460 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.848+0000 7efc24c00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7efc0c0b5490 0x7efc0c0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.848+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7efc181072e0 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.848+0000 7efc27400640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7efc0c0b5490 0x7efc0c0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.848+0000 7efc24c00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] conn(0x7efc0c0c0550 0x7efc0c0c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.848+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 --> [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc0c0c30c0 con 0x7efc0c0c0550 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.848+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7efc18107690 con 0x7efc28102c10 2024-09-11T00:22:11.875 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.849+0000 7efc2ce00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] conn(0x7efc0c0c0550 0x7efc0c0c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.849+0000 7efc27400640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7efc0c0b5490 0x7efc0c0b7950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efc140095f0 tx=0x7efc14005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.849+0000 7efc2ce00640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] conn(0x7efc0c0c0550 0x7efc0c0c29f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.850+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== osd.0 v2:172.21.15.103:6802/2122983422 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc0c0c30c0 con 0x7efc0c0c0550 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.864+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 --> [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efc28063be0 con 0x7efc0c0c0550 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.864+0000 7efc24c00640 1 -- 172.21.15.103:0/3628343776 <== osd.0 v2:172.21.15.103:6802/2122983422 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efc28063be0 con 0x7efc0c0c0550 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] conn(0x7efc0c0c0550 msgr2=0x7efc0c0c29f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] conn(0x7efc0c0c0550 0x7efc0c0c29f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7efc0c0b5490 msgr2=0x7efc0c0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7efc0c0b5490 0x7efc0c0b7950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efc140095f0 tx=0x7efc14005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:11.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc28102c10 msgr2=0x7efc2819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:11.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 --2- 172.21.15.103:0/3628343776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc28102c10 0x7efc2819b390 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efc18005ab0 tx=0x7efc18003c10 comp rx=0 tx=0).stop 2024-09-11T00:22:11.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2ce00640 1 -- 172.21.15.103:0/3628343776 reap_dead start 2024-09-11T00:22:11.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 shutdown_connections 2024-09-11T00:22:11.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 >> 172.21.15.103:0/3628343776 conn(0x7efc280fe660 msgr2=0x7efc28075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:11.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 shutdown_connections 2024-09-11T00:22:11.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:11.865+0000 7efc2d800640 1 -- 172.21.15.103:0/3628343776 wait complete. 2024-09-11T00:22:12.008 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:12.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:12 smithi103 ceph-mon[30578]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:12.739 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:12 smithi117 ceph-mon[35525]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:12.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:12 smithi183 ceph-mon[35934]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:13.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 -- 172.21.15.103:0/3673702441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc10f3b0 msgr2=0x7effdc1117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 --2- 172.21.15.103:0/3673702441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc10f3b0 0x7effdc1117a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7effd4009900 tx=0x7effd402a410 comp rx=0 tx=0).stop 2024-09-11T00:22:13.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 -- 172.21.15.103:0/3673702441 shutdown_connections 2024-09-11T00:22:13.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 --2- 172.21.15.103:0/3673702441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc10f3b0 0x7effdc1117a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 --2- 172.21.15.103:0/3673702441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effdc075de0 0x7effdc10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 --2- 172.21.15.103:0/3673702441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc0754c0 0x7effdc0758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 -- 172.21.15.103:0/3673702441 >> 172.21.15.103:0/3673702441 conn(0x7effdc0fe810 msgr2=0x7effdc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 -- 172.21.15.103:0/3673702441 shutdown_connections 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.110+0000 7effe0e00640 1 -- 172.21.15.103:0/3673702441 wait complete. 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 Processor -- start 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 -- start start 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effdc0754c0 0x7effdc19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc075de0 0x7effdc19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc10f3b0 0x7effdc1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7effdc114fd0 con 0x7effdc075de0 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7effdc114e50 con 0x7effdc0754c0 2024-09-11T00:22:13.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.111+0000 7effe0e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7effdc115150 con 0x7effdc10f3b0 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc075de0 0x7effdc19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc075de0 0x7effdc19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47382/0 (socket says 172.21.15.103:47382) 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 -- 172.21.15.103:0/1053984743 learned_addr learned my addr 172.21.15.103:0/1053984743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effda800640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc10f3b0 0x7effdc1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effdc0754c0 0x7effdc19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 -- 172.21.15.103:0/1053984743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc10f3b0 msgr2=0x7effdc1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7effdc10f3b0 0x7effdc1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 -- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effdc0754c0 msgr2=0x7effdc19b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effdc0754c0 0x7effdc19b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 -- 172.21.15.103:0/1053984743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc1a1570 con 0x7effdc075de0 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9e00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effdc0754c0 0x7effdc19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.112+0000 7effd9400640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc075de0 0x7effdc19ba00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effc800ea60 tx=0x7effc800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:13.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.113+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc800cde0 con 0x7effdc075de0 2024-09-11T00:22:13.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.113+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effc80035f0 con 0x7effdc075de0 2024-09-11T00:22:13.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.113+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc1a1860 con 0x7effdc075de0 2024-09-11T00:22:13.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.113+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8010690 con 0x7effdc075de0 2024-09-11T00:22:13.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.113+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc1a7f20 con 0x7effdc075de0 2024-09-11T00:22:13.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.114+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7effdc100730 con 0x7effdc075de0 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.116+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7effc8004000 con 0x7effdc075de0 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.118+0000 7effd2a00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effc00b53f0 0x7effc00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.118+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7effc80e6730 con 0x7effdc075de0 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.118+0000 7effd9e00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effc00b53f0 0x7effc00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.118+0000 7effd2a00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] conn(0x7effc00c04b0 0x7effc00c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.118+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 --> [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7effc8025ea0 con 0x7effc00c04b0 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.118+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7effc80e6ae0 con 0x7effdc075de0 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.119+0000 7effda800640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] conn(0x7effc00c04b0 0x7effc00c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.119+0000 7effd9e00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effc00b53f0 0x7effc00b78b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7effcc005fd0 tx=0x7effcc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:13.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.120+0000 7effda800640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] conn(0x7effc00c04b0 0x7effc00c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:13.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.121+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== osd.10 v2:172.21.15.117:6824/2500948230 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7effc8025ea0 con 0x7effc00c04b0 2024-09-11T00:22:13.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.136+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 --> [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7effdc0769d0 con 0x7effc00c04b0 2024-09-11T00:22:13.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.136+0000 7effd2a00640 1 -- 172.21.15.103:0/1053984743 <== osd.10 v2:172.21.15.117:6824/2500948230 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7effdc0769d0 con 0x7effc00c04b0 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] conn(0x7effc00c04b0 msgr2=0x7effc00c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] conn(0x7effc00c04b0 0x7effc00c2950 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effc00b53f0 msgr2=0x7effc00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effc00b53f0 0x7effc00b78b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7effcc005fd0 tx=0x7effcc009500 comp rx=0 tx=0).stop 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc075de0 msgr2=0x7effdc19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 --2- 172.21.15.103:0/1053984743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effdc075de0 0x7effdc19ba00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7effc800ea60 tx=0x7effc800ef20 comp rx=0 tx=0).stop 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effda800640 1 -- 172.21.15.103:0/1053984743 reap_dead start 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 shutdown_connections 2024-09-11T00:22:13.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 >> 172.21.15.103:0/1053984743 conn(0x7effdc0fe810 msgr2=0x7effdc1115a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:13.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 shutdown_connections 2024-09-11T00:22:13.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.137+0000 7effe0e00640 1 -- 172.21.15.103:0/1053984743 wait complete. 2024-09-11T00:22:13.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 -- 172.21.15.103:0/972097209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 msgr2=0x7f62e8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 --2- 172.21.15.103:0/972097209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e8075920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009900 tx=0x7f62d802a410 comp rx=0 tx=0).stop 2024-09-11T00:22:13.764 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 -- 172.21.15.103:0/972097209 shutdown_connections 2024-09-11T00:22:13.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 --2- 172.21.15.103:0/972097209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e8113b20 0x7f62e8115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 --2- 172.21.15.103:0/972097209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e8075e60 0x7f62e81135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 --2- 172.21.15.103:0/972097209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e8075920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 -- 172.21.15.103:0/972097209 >> 172.21.15.103:0/972097209 conn(0x7f62e80fe830 msgr2=0x7f62e8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:13.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 -- 172.21.15.103:0/972097209 shutdown_connections 2024-09-11T00:22:13.765 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.762+0000 7f62eca00640 1 -- 172.21.15.103:0/972097209 wait complete. 2024-09-11T00:22:13.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.763+0000 7f62eca00640 1 Processor -- start 2024-09-11T00:22:13.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.763+0000 7f62eca00640 1 -- start start 2024-09-11T00:22:13.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62eca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62eca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e8075e60 0x7f62e819fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.766 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62eca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e8113b20 0x7f62e81a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62eca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62e81193a0 con 0x7f62e8113b20 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62eca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f62e8119220 con 0x7f62e8075e60 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62eca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62e8119520 con 0x7f62e8075540 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62e5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62e5e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:60596/0 (socket says 172.21.15.103:60596) 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62e5e00640 1 -- 172.21.15.103:0/119684058 learned_addr learned my addr 172.21.15.103:0/119684058 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62e6800640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e8113b20 0x7f62e81a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.767 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.764+0000 7f62e5400640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e8075e60 0x7f62e819fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62e5400640 1 -- 172.21.15.103:0/119684058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 msgr2=0x7f62e819f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62e5400640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e819f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62e5400640 1 -- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e8113b20 msgr2=0x7f62e81a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62e5400640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e8113b20 0x7f62e81a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62e5400640 1 -- 172.21.15.103:0/119684058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e81a5980 con 0x7f62e8075e60 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62e5e00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62e5400640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e8075e60 0x7f62e819fe10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.765+0000 7f62dea00640 1 -- 172.21.15.103:0/119684058 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d400ccc0 con 0x7f62e8075e60 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.766+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e81a5c70 con 0x7f62e8075e60 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.766+0000 7f62dea00640 1 -- 172.21.15.103:0/119684058 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62d40108e0 con 0x7f62e8075e60 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.766+0000 7f62dea00640 1 -- 172.21.15.103:0/119684058 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4018880 con 0x7f62e8075e60 2024-09-11T00:22:13.768 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.766+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e81ac330 con 0x7f62e8075e60 2024-09-11T00:22:13.769 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.767+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e8075920 con 0x7f62e8075e60 2024-09-11T00:22:13.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.771+0000 7f62dea00640 1 -- 172.21.15.103:0/119684058 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f62d4010430 con 0x7f62e8075e60 2024-09-11T00:22:13.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.773+0000 7f62dea00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62cc0b5370 0x7f62cc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.773+0000 7f62dea00640 1 -- 172.21.15.103:0/119684058 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f62d40e66b0 con 0x7f62e8075e60 2024-09-11T00:22:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.773+0000 7f62e5e00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62cc0b5370 0x7f62cc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:13.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.773+0000 7f62dea00640 1 -- 172.21.15.103:0/119684058 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f62d40a8bc0 con 0x7f62e8075e60 2024-09-11T00:22:13.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.774+0000 7f62e5e00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62cc0b5370 0x7f62cc0b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005ab0 tx=0x7f62d8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:13.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.981+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f62e81ac6a0 con 0x7f62e8075e60 2024-09-11T00:22:13.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.982+0000 7f62dea00640 1 -- 172.21.15.103:0/119684058 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f62d40ad050 con 0x7f62e8075e60 2024-09-11T00:22:13.984 INFO:teuthology.orchestra.run.smithi103.stdout:55834574937 2024-09-11T00:22:13.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62cc0b5370 msgr2=0x7f62cc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62cc0b5370 0x7f62cc0b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f62d8005ab0 tx=0x7f62d8005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:13.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e8075e60 msgr2=0x7f62e819fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:13.987 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e8075e60 0x7f62e819fe10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f62d400e9f0 tx=0x7f62d400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 shutdown_connections 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f62cc0b5370 0x7f62cc0b7830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e8113b20 0x7f62e81a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e8075e60 0x7f62e819fe10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 --2- 172.21.15.103:0/119684058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e8075540 0x7f62e819f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 >> 172.21.15.103:0/119684058 conn(0x7f62e80fe830 msgr2=0x7f62e80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.986+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 shutdown_connections 2024-09-11T00:22:13.988 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.986+0000 7f62eca00640 1 -- 172.21.15.103:0/119684058 wait complete. 2024-09-11T00:22:14.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 -- 172.21.15.103:0/1899860601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d58104fc0 msgr2=0x7f7d581053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 --2- 172.21.15.103:0/1899860601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d58104fc0 0x7f7d581053a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d48009900 tx=0x7f7d4802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 -- 172.21.15.103:0/1899860601 shutdown_connections 2024-09-11T00:22:14.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 --2- 172.21.15.103:0/1899860601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d5810b350 0x7f7d58111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 --2- 172.21.15.103:0/1899860601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5810a9d0 0x7f7d5810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.042 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 --2- 172.21.15.103:0/1899860601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d58104fc0 0x7f7d581053a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 -- 172.21.15.103:0/1899860601 >> 172.21.15.103:0/1899860601 conn(0x7f7d581009f0 msgr2=0x7f7d58102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.984+0000 7f7d5d800640 1 -- 172.21.15.103:0/1899860601 shutdown_connections 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.985+0000 7f7d5d800640 1 -- 172.21.15.103:0/1899860601 wait complete. 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.986+0000 7f7d5d800640 1 Processor -- start 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.986+0000 7f7d5d800640 1 -- start start 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d5d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d58104fc0 0x7f7d5819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d5d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5810a9d0 0x7f7d5819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d5d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5810b350 0x7f7d581a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d5d800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58114fd0 con 0x7f7d58104fc0 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d5d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58114e50 con 0x7f7d5810b350 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d5d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58115150 con 0x7f7d5810a9d0 2024-09-11T00:22:14.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5810a9d0 0x7f7d5819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5810a9d0 0x7f7d5819b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:60608/0 (socket says 172.21.15.103:60608) 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 -- 172.21.15.103:0/4269629663 learned_addr learned my addr 172.21.15.103:0/4269629663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d57400640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5810b350 0x7f7d581a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 -- 172.21.15.103:0/4269629663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5810b350 msgr2=0x7f7d581a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5810b350 0x7f7d581a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 -- 172.21.15.103:0/4269629663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d58104fc0 msgr2=0x7f7d5819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d58104fc0 0x7f7d5819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.987+0000 7f7d56000640 1 -- 172.21.15.103:0/4269629663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d581a1530 con 0x7f7d5810a9d0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.988+0000 7f7d56000640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5810a9d0 0x7f7d5819b9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e9f0 tx=0x7f7d4400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.988+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400cd80 con 0x7f7d5810a9d0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.988+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d581a1820 con 0x7f7d5810a9d0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.988+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d440054d0 con 0x7f7d5810a9d0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.988+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d44010690 con 0x7f7d5810a9d0 2024-09-11T00:22:14.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.988+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d581a7ee0 con 0x7f7d5810a9d0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.989+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7d580638e0 con 0x7f7d5810a9d0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.992+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f7d44004e90 con 0x7f7d5810a9d0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.994+0000 7f7d4f400640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7d3c0b5370 0x7f7d3c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.994+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7d440e6700 con 0x7f7d5810a9d0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.994+0000 7f7d56a00640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7d3c0b5370 0x7f7d3c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.994+0000 7f7d4f400640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] conn(0x7f7d3c0c0430 0x7f7d3c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.994+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 --> [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7d4400be50 con 0x7f7d3c0c0430 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.994+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f7d440e6ab0 con 0x7f7d5810a9d0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.995+0000 7f7d57400640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] conn(0x7f7d3c0c0430 0x7f7d3c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.995+0000 7f7d56a00640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7d3c0b5370 0x7f7d3c0b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d4802a910 tx=0x7f7d48031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.995+0000 7f7d57400640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] conn(0x7f7d3c0c0430 0x7f7d3c0c28d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.996+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== osd.8 v2:172.21.15.183:6816/3906862108 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7d4400be50 con 0x7f7d3c0c0430 2024-09-11T00:22:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.011+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 --> [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7d58063d60 con 0x7f7d3c0c0430 2024-09-11T00:22:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.012+0000 7f7d4f400640 1 -- 172.21.15.103:0/4269629663 <== osd.8 v2:172.21.15.183:6816/3906862108 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7d58063d60 con 0x7f7d3c0c0430 2024-09-11T00:22:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] conn(0x7f7d3c0c0430 msgr2=0x7f7d3c0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] conn(0x7f7d3c0c0430 0x7f7d3c0c28d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7d3c0b5370 msgr2=0x7f7d3c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7d3c0b5370 0x7f7d3c0b7830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7d4802a910 tx=0x7f7d48031400 comp rx=0 tx=0).stop 2024-09-11T00:22:14.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5810a9d0 msgr2=0x7f7d5819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 --2- 172.21.15.103:0/4269629663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d5810a9d0 0x7f7d5819b9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e9f0 tx=0x7f7d4400edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d57400640 1 -- 172.21.15.103:0/4269629663 reap_dead start 2024-09-11T00:22:14.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 shutdown_connections 2024-09-11T00:22:14.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.013+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 >> 172.21.15.103:0/4269629663 conn(0x7f7d581009f0 msgr2=0x7f7d58102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.014+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 shutdown_connections 2024-09-11T00:22:14.047 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:14.014+0000 7f7d5d800640 1 -- 172.21.15.103:0/4269629663 wait complete. 2024-09-11T00:22:14.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:13 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3149067293' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:22:14.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:13 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3149067293' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:22:14.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:13 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3149067293' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 -- 172.21.15.103:0/1199534928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 msgr2=0x7fd4ec108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 --2- 172.21.15.103:0/1199534928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 0x7fd4ec108c10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4009900 tx=0x7fd4d402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 -- 172.21.15.103:0/1199534928 shutdown_connections 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 --2- 172.21.15.103:0/1199534928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec109150 0x7fd4ec10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 --2- 172.21.15.103:0/1199534928 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 0x7fd4ec108c10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 --2- 172.21.15.103:0/1199534928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4ec102dc0 0x7fd4ec1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 -- 172.21.15.103:0/1199534928 >> 172.21.15.103:0/1199534928 conn(0x7fd4ec0fe830 msgr2=0x7fd4ec100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 -- 172.21.15.103:0/1199534928 shutdown_connections 2024-09-11T00:22:14.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.602+0000 7fd4f3400640 1 -- 172.21.15.103:0/1199534928 wait complete. 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.603+0000 7fd4f3400640 1 Processor -- start 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.603+0000 7fd4f3400640 1 -- start start 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec102dc0 0x7fd4ec19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 0x7fd4ec19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4ec109150 0x7fd4ec1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f3400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec112e10 con 0x7fd4ec1087d0 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f3400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec112c90 con 0x7fd4ec102dc0 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f3400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec112f90 con 0x7fd4ec109150 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 0x7fd4ec19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec102dc0 0x7fd4ec19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4e3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 0x7fd4ec19ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47400/0 (socket says 172.21.15.103:47400) 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4e3e00640 1 -- 172.21.15.103:0/3149067293 learned_addr learned my addr 172.21.15.103:0/3149067293 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:14.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f0a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec102dc0 0x7fd4ec19b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:60638/0 (socket says 172.21.15.103:60638) 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.604+0000 7fd4f1400640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4ec109150 0x7fd4ec1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f0a00640 1 -- 172.21.15.103:0/3149067293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4ec109150 msgr2=0x7fd4ec1a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f0a00640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4ec109150 0x7fd4ec1a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f0a00640 1 -- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 msgr2=0x7fd4ec19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f0a00640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 0x7fd4ec19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f0a00640 1 -- 172.21.15.103:0/3149067293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec1a15f0 con 0x7fd4ec102dc0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f0a00640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec102dc0 0x7fd4ec19b540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005fd0 tx=0x7fd4dc00dd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4ea000640 1 -- 172.21.15.103:0/3149067293 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc009c30 con 0x7fd4ec102dc0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec1a18e0 con 0x7fd4ec102dc0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4ea000640 1 -- 172.21.15.103:0/3149067293 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4dc00be30 con 0x7fd4ec102dc0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4ea000640 1 -- 172.21.15.103:0/3149067293 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc003da0 con 0x7fd4ec102dc0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.605+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec1a8170 con 0x7fd4ec102dc0 2024-09-11T00:22:14.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.606+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec104070 con 0x7fd4ec102dc0 2024-09-11T00:22:14.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.609+0000 7fd4ea000640 1 -- 172.21.15.103:0/3149067293 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fd4dc003f40 con 0x7fd4ec102dc0 2024-09-11T00:22:14.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.610+0000 7fd4ea000640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4d00b5370 0x7fd4d00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:14.448 INFO:teuthology.orchestra.run.smithi103.stdout:133143986250 2024-09-11T00:22:14.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.610+0000 7fd4ea000640 1 -- 172.21.15.103:0/3149067293 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd4dc0e5ee0 con 0x7fd4ec102dc0 2024-09-11T00:22:14.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.610+0000 7fd4e3e00640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4d00b5370 0x7fd4d00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:14.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.611+0000 7fd4e3e00640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4d00b5370 0x7fd4d00b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4031440 tx=0x7fd4d40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:14.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.611+0000 7fd4ea000640 1 -- 172.21.15.103:0/3149067293 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd4dc017020 con 0x7fd4ec102dc0 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.816+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd4ec10f340 con 0x7fd4ec102dc0 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.817+0000 7fd4ea000640 1 -- 172.21.15.103:0/3149067293 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd4dc0ac880 con 0x7fd4ec102dc0 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4d00b5370 msgr2=0x7fd4d00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4d00b5370 0x7fd4d00b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4031440 tx=0x7fd4d40058d0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec102dc0 msgr2=0x7fd4ec19b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec102dc0 0x7fd4ec19b540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc005fd0 tx=0x7fd4dc00dd20 comp rx=0 tx=0).stop 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 shutdown_connections 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4d00b5370 0x7fd4d00b7830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4ec109150 0x7fd4ec1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd4ec1087d0 0x7fd4ec19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 --2- 172.21.15.103:0/3149067293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4ec102dc0 0x7fd4ec19b540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.819+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 >> 172.21.15.103:0/3149067293 conn(0x7fd4ec0fe830 msgr2=0x7fd4ec104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:14.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.820+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 shutdown_connections 2024-09-11T00:22:14.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:13.820+0000 7fd4f3400640 1 -- 172.21.15.103:0/3149067293 wait complete. 2024-09-11T00:22:15.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:14 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/119684058' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:22:15.132 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:14 smithi103 ceph-mon[30578]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T00:22:15.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:14 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/119684058' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:22:15.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:14 smithi117 ceph-mon[35525]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T00:22:15.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:14 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/119684058' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-11T00:22:15.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:14 smithi183 ceph-mon[35934]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 -- 172.21.15.103:0/990612412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c010f3e0 msgr2=0x7fb1c01117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 --2- 172.21.15.103:0/990612412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c010f3e0 0x7fb1c01117d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009860 tx=0x7fb1b802a340 comp rx=0 tx=0).stop 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 -- 172.21.15.103:0/990612412 shutdown_connections 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 --2- 172.21.15.103:0/990612412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c010f3e0 0x7fb1c01117d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 --2- 172.21.15.103:0/990612412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c0075f40 0x7fb1c010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 --2- 172.21.15.103:0/990612412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1c0075540 0x7fb1c0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 -- 172.21.15.103:0/990612412 >> 172.21.15.103:0/990612412 conn(0x7fb1c00fe830 msgr2=0x7fb1c0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 -- 172.21.15.103:0/990612412 shutdown_connections 2024-09-11T00:22:15.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.637+0000 7fb1c6e00640 1 -- 172.21.15.103:0/990612412 wait complete. 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 Processor -- start 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 -- start start 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0075540 0x7fb1c019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1c0075f40 0x7fb1c019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c010f3e0 0x7fb1c01a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0114f80 con 0x7fb1c0075540 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0114e00 con 0x7fb1c010f3e0 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1c6e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0115100 con 0x7fb1c0075f40 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0075540 0x7fb1c019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1bfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0075540 0x7fb1c019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47462/0 (socket says 172.21.15.103:47462) 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.638+0000 7fb1bfe00640 1 -- 172.21.15.103:0/2065089835 learned_addr learned my addr 172.21.15.103:0/2065089835 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bf400640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1c0075f40 0x7fb1c019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:15.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1c4e00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c010f3e0 0x7fb1c01a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bfe00640 1 -- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1c0075f40 msgr2=0x7fb1c019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bfe00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb1c0075f40 0x7fb1c019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bfe00640 1 -- 172.21.15.103:0/2065089835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c010f3e0 msgr2=0x7fb1c01a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bfe00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1c010f3e0 0x7fb1c01a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bfe00640 1 -- 172.21.15.103:0/2065089835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c01a1520 con 0x7fb1c0075540 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bfe00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0075540 0x7fb1c019b470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000eb90 tx=0x7fb1b000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0004d20 con 0x7fb1c0075540 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1b000ee90 con 0x7fb1c0075540 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.639+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0010670 con 0x7fb1c0075540 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.640+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c01a1810 con 0x7fb1c0075540 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.640+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c01a7ed0 con 0x7fb1c0075540 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.641+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1c0100ea0 con 0x7fb1c0075540 2024-09-11T00:22:15.661 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.643+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fb1b0005000 con 0x7fb1c0075540 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.644+0000 7fb1bcc00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb1a40b53f0 0x7fb1a40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.644+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fb1b00e6ad0 con 0x7fb1c0075540 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.644+0000 7fb1bf400640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb1a40b53f0 0x7fb1a40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.645+0000 7fb1bcc00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] conn(0x7fb1a40c04b0 0x7fb1a40c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.645+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 --> [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1a40c3020 con 0x7fb1a40c04b0 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.645+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fb1b00e6e80 con 0x7fb1c0075540 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.645+0000 7fb1bf400640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb1a40b53f0 0x7fb1a40b78b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb1c01109f0 tx=0x7fb1ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.645+0000 7fb1c4e00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] conn(0x7fb1a40c04b0 0x7fb1a40c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.646+0000 7fb1c4e00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] conn(0x7fb1a40c04b0 0x7fb1a40c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:15.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.647+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== osd.5 v2:172.21.15.183:6808/660429502 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1a40c3020 con 0x7fb1a40c04b0 2024-09-11T00:22:15.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.661+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 --> [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1c0076a50 con 0x7fb1a40c04b0 2024-09-11T00:22:15.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1bcc00640 1 -- 172.21.15.103:0/2065089835 <== osd.5 v2:172.21.15.183:6808/660429502 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb1c0076a50 con 0x7fb1a40c04b0 2024-09-11T00:22:15.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] conn(0x7fb1a40c04b0 msgr2=0x7fb1a40c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:15.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1c6e00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] conn(0x7fb1a40c04b0 0x7fb1a40c2950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:15.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb1a40b53f0 msgr2=0x7fb1a40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:15.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1c6e00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb1a40b53f0 0x7fb1a40b78b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb1c01109f0 tx=0x7fb1ac009260 comp rx=0 tx=0).stop 2024-09-11T00:22:15.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0075540 msgr2=0x7fb1c019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:15.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1c6e00640 1 --2- 172.21.15.103:0/2065089835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1c0075540 0x7fb1c019b470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000eb90 tx=0x7fb1b000c6a0 comp rx=0 tx=0).stop 2024-09-11T00:22:15.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.662+0000 7fb1c4e00640 1 -- 172.21.15.103:0/2065089835 reap_dead start 2024-09-11T00:22:15.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.663+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 shutdown_connections 2024-09-11T00:22:15.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.663+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 >> 172.21.15.103:0/2065089835 conn(0x7fb1c00fe830 msgr2=0x7fb1c00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:15.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.663+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 shutdown_connections 2024-09-11T00:22:15.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:15.663+0000 7fb1c6e00640 1 -- 172.21.15.103:0/2065089835 wait complete. 2024-09-11T00:22:16.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986248 got 133143986250 for osd.4 2024-09-11T00:22:16.421 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:16.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:16 smithi103 ceph-mon[30578]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:16 smithi183 ceph-mon[35934]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:16.932 INFO:teuthology.orchestra.run.smithi103.stdout:34359738464 2024-09-11T00:22:16.933 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T00:22:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:16 smithi117 ceph-mon[35525]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:17.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 -- 172.21.15.103:0/1558011319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec10a820 msgr2=0x7f7cec10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 --2- 172.21.15.103:0/1558011319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec10a820 0x7f7cec10ac60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009900 tx=0x7f7cd802a400 comp rx=0 tx=0).stop 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 -- 172.21.15.103:0/1558011319 shutdown_connections 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 --2- 172.21.15.103:0/1558011319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cec10b1a0 0x7f7cec1119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 --2- 172.21.15.103:0/1558011319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec10a820 0x7f7cec10ac60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 --2- 172.21.15.103:0/1558011319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cec104e10 0x7f7cec1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 -- 172.21.15.103:0/1558011319 >> 172.21.15.103:0/1558011319 conn(0x7f7cec100840 msgr2=0x7f7cec102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 -- 172.21.15.103:0/1558011319 shutdown_connections 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.460+0000 7f7cf1000640 1 -- 172.21.15.103:0/1558011319 wait complete. 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.461+0000 7f7cf1000640 1 Processor -- start 2024-09-11T00:22:17.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.461+0000 7f7cf1000640 1 -- start start 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cf1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec104e10 0x7f7cec19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cf1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cec10a820 0x7f7cec19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cf1000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cec10b1a0 0x7f7cec1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cf1000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec114e70 con 0x7f7cec104e10 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cf1000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec114cf0 con 0x7f7cec10a820 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cf1000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec114ff0 con 0x7f7cec10b1a0 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec104e10 0x7f7cec19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec104e10 0x7f7cec19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47484/0 (socket says 172.21.15.103:47484) 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 -- 172.21.15.103:0/2502447773 learned_addr learned my addr 172.21.15.103:0/2502447773 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceaa00640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cec10a820 0x7f7cec19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cebe00640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cec10b1a0 0x7f7cec1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 -- 172.21.15.103:0/2502447773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cec10b1a0 msgr2=0x7f7cec1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cec10b1a0 0x7f7cec1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 -- 172.21.15.103:0/2502447773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cec10a820 msgr2=0x7f7cec19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cec10a820 0x7f7cec19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7ceb400640 1 -- 172.21.15.103:0/2502447773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec1a1350 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.462+0000 7f7cebe00640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cec10b1a0 0x7f7cec1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.463+0000 7f7ceb400640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec104e10 0x7f7cec19b2a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00d770 tx=0x7f7cdc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.463+0000 7f7ce3e00640 1 -- 172.21.15.103:0/2502447773 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc017070 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.463+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1a1640 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.463+0000 7f7ce3e00640 1 -- 172.21.15.103:0/2502447773 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cdc0054d0 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.463+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1a7d00 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.463+0000 7f7ce3e00640 1 -- 172.21.15.103:0/2502447773 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc020060 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.464+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec1051f0 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.466+0000 7f7ce3e00640 1 -- 172.21.15.103:0/2502447773 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f7cdc023020 con 0x7f7cec104e10 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.469+0000 7f7ce3e00640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7cd00b5410 0x7f7cd00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.469+0000 7f7ce3e00640 1 -- 172.21.15.103:0/2502447773 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7cdc0e5990 con 0x7f7cec104e10 2024-09-11T00:22:17.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.469+0000 7f7ceaa00640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7cd00b5410 0x7f7cd00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.470+0000 7f7ceaa00640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7cd00b5410 0x7f7cd00b78d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8005ab0 tx=0x7f7cd8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:17.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.473+0000 7f7ce3e00640 1 -- 172.21.15.103:0/2502447773 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7cdc0ac330 con 0x7f7cec104e10 2024-09-11T00:22:17.670 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.669+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f7cec10c9f0 con 0x7f7cec104e10 2024-09-11T00:22:17.671 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.669+0000 7f7ce3e00640 1 -- 172.21.15.103:0/2502447773 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7cdc0abc30 con 0x7f7cec104e10 2024-09-11T00:22:17.671 INFO:teuthology.orchestra.run.smithi103.stdout:107374182480 2024-09-11T00:22:17.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7cd00b5410 msgr2=0x7f7cd00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7cd00b5410 0x7f7cd00b78d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8005ab0 tx=0x7f7cd8005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:17.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec104e10 msgr2=0x7f7cec19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec104e10 0x7f7cec19b2a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00d770 tx=0x7f7cdc00dc30 comp rx=0 tx=0).stop 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 shutdown_connections 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7cd00b5410 0x7f7cd00b78d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7cec10b1a0 0x7f7cec1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cec10a820 0x7f7cec19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 --2- 172.21.15.103:0/2502447773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cec104e10 0x7f7cec19b2a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.671+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 >> 172.21.15.103:0/2502447773 conn(0x7f7cec100840 msgr2=0x7f7cec102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.672+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 shutdown_connections 2024-09-11T00:22:17.674 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.672+0000 7f7cf1000640 1 -- 172.21.15.103:0/2502447773 wait complete. 2024-09-11T00:22:17.989 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:17 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2502447773' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:22:17.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 -- 172.21.15.103:0/298109626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a68108620 msgr2=0x7f0a68108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 --2- 172.21.15.103:0/298109626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a68108620 0x7f0a68108a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0a50009900 tx=0x7f0a5002a410 comp rx=0 tx=0).stop 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 -- 172.21.15.103:0/298109626 shutdown_connections 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 --2- 172.21.15.103:0/298109626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a68108fa0 0x7f0a6810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 --2- 172.21.15.103:0/298109626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a68108620 0x7f0a68108a60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 --2- 172.21.15.103:0/298109626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a68102c10 0x7f0a68102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 -- 172.21.15.103:0/298109626 >> 172.21.15.103:0/298109626 conn(0x7f0a680fe6c0 msgr2=0x7f0a68100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 -- 172.21.15.103:0/298109626 shutdown_connections 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.987+0000 7f0a67e00640 1 -- 172.21.15.103:0/298109626 wait complete. 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.988+0000 7f0a67e00640 1 Processor -- start 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.988+0000 7f0a67e00640 1 -- start start 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a68102c10 0x7f0a6819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a68108620 0x7f0a6819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a67e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a68108fa0 0x7f0a681a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a67e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68112c70 con 0x7f0a68108fa0 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a67e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68112af0 con 0x7f0a68108620 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a67e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68112df0 con 0x7f0a68102c10 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a68102c10 0x7f0a6819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a67400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a68108fa0 0x7f0a681a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a66a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a68102c10 0x7f0a6819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:60678/0 (socket says 172.21.15.103:60678) 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a66a00640 1 -- 172.21.15.103:0/531642945 learned_addr learned my addr 172.21.15.103:0/531642945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a66000640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a68108620 0x7f0a6819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.989+0000 7f0a66a00640 1 -- 172.21.15.103:0/531642945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a68108620 msgr2=0x7f0a6819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a66a00640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a68108620 0x7f0a6819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a66a00640 1 -- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a68108fa0 msgr2=0x7f0a681a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:17.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a66a00640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a68108fa0 0x7f0a681a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a66a00640 1 -- 172.21.15.103:0/531642945 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a681a13e0 con 0x7f0a68102c10 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a66a00640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a68102c10 0x7f0a6819b330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800ebc0 tx=0x7f0a5800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58010630 con 0x7f0a68102c10 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a681a16d0 con 0x7f0a68102c10 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a58010c80 con 0x7f0a68102c10 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a58018b60 con 0x7f0a68102c10 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.990+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a681a7f60 con 0x7f0a68102c10 2024-09-11T00:22:17.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.991+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a680638e0 con 0x7f0a68102c10 2024-09-11T00:22:17.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.994+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f0a580107d0 con 0x7f0a68102c10 2024-09-11T00:22:17.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.995+0000 7f0a5f400640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0a4c0b5490 0x7f0a4c0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.995+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f0a580e67b0 con 0x7f0a68102c10 2024-09-11T00:22:17.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.995+0000 7f0a66000640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0a4c0b5490 0x7f0a4c0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.995+0000 7f0a5f400640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] conn(0x7f0a4c0c0550 0x7f0a4c0c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:17.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.995+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 --> [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a4c0c30c0 con 0x7f0a4c0c0550 2024-09-11T00:22:17.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.995+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0a580e6b60 con 0x7f0a68102c10 2024-09-11T00:22:17.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.996+0000 7f0a67400640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] conn(0x7f0a4c0c0550 0x7f0a4c0c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:17.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.996+0000 7f0a66000640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0a4c0b5490 0x7f0a4c0b7950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005ab0 tx=0x7f0a50005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:17.998 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.996+0000 7f0a67400640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] conn(0x7f0a4c0c0550 0x7f0a4c0c29f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:17.999 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:17.997+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== osd.9 v2:172.21.15.103:6826/64454554 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a4c0c30c0 con 0x7f0a4c0c0550 2024-09-11T00:22:18.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.011+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 --> [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a68063db0 con 0x7f0a4c0c0550 2024-09-11T00:22:18.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a5f400640 1 -- 172.21.15.103:0/531642945 <== osd.9 v2:172.21.15.103:6826/64454554 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0a68063db0 con 0x7f0a4c0c0550 2024-09-11T00:22:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] conn(0x7f0a4c0c0550 msgr2=0x7f0a4c0c29f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] conn(0x7f0a4c0c0550 0x7f0a4c0c29f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0a4c0b5490 msgr2=0x7f0a4c0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0a4c0b5490 0x7f0a4c0b7950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0a50005ab0 tx=0x7f0a50005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a68102c10 msgr2=0x7f0a6819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 --2- 172.21.15.103:0/531642945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0a68102c10 0x7f0a6819b330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0a5800ebc0 tx=0x7f0a5800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:22:18.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67400640 1 -- 172.21.15.103:0/531642945 reap_dead start 2024-09-11T00:22:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 shutdown_connections 2024-09-11T00:22:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.012+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 >> 172.21.15.103:0/531642945 conn(0x7f0a680fe6c0 msgr2=0x7f0a68105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.013+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 shutdown_connections 2024-09-11T00:22:18.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:18.013+0000 7f0a67e00640 1 -- 172.21.15.103:0/531642945 wait complete. 2024-09-11T00:22:18.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:17 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2502447773' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:22:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:17 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2502447773' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-11T00:22:18.916 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:18 smithi103 ceph-mon[30578]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T00:22:19.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:18 smithi183 ceph-mon[35934]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T00:22:19.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:18 smithi117 ceph-mon[35525]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-11T00:22:19.410 INFO:teuthology.orchestra.run.smithi103.stdout:227633266740 2024-09-11T00:22:19.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T00:22:19.772 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574935 got 55834574937 for osd.1 2024-09-11T00:22:19.772 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:20.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.271+0000 7ff2a8c00640 1 -- 172.21.15.103:0/2639779752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4101220 msgr2=0x7ff2a4101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:20.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.271+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/2639779752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4101220 0x7ff2a4101600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff294009900 tx=0x7ff29402a410 comp rx=0 tx=0).stop 2024-09-11T00:22:20.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.272+0000 7ff2a8c00640 1 -- 172.21.15.103:0/2639779752 shutdown_connections 2024-09-11T00:22:20.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.272+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/2639779752 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4105ef0 0x7ff2a410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.272+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/2639779752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4105570 0x7ff2a41059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.272+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/2639779752 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4101220 0x7ff2a4101600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.272+0000 7ff2a8c00640 1 -- 172.21.15.103:0/2639779752 >> 172.21.15.103:0/2639779752 conn(0x7ff2a4078100 msgr2=0x7ff2a40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:20.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.272+0000 7ff2a8c00640 1 -- 172.21.15.103:0/2639779752 shutdown_connections 2024-09-11T00:22:20.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.272+0000 7ff2a8c00640 1 -- 172.21.15.103:0/2639779752 wait complete. 2024-09-11T00:22:20.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 Processor -- start 2024-09-11T00:22:20.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 -- start start 2024-09-11T00:22:20.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4101220 0x7ff2a419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:20.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4105570 0x7ff2a419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:20.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4105ef0 0x7ff2a41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:20.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4112e40 con 0x7ff2a4105ef0 2024-09-11T00:22:20.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4112cc0 con 0x7ff2a4105570 2024-09-11T00:22:20.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a8c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a4112fc0 con 0x7ff2a4101220 2024-09-11T00:22:20.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4105ef0 0x7ff2a41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:20.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4101220 0x7ff2a419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:20.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4105ef0 0x7ff2a41a0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:47512/0 (socket says 172.21.15.103:47512) 2024-09-11T00:22:20.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4101220 0x7ff2a419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:60698/0 (socket says 172.21.15.103:60698) 2024-09-11T00:22:20.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.273+0000 7ff2a2800640 1 -- 172.21.15.103:0/1550969313 learned_addr learned my addr 172.21.15.103:0/1550969313 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:20.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff299400640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4105570 0x7ff2a419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:20.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff2a1e00640 1 -- 172.21.15.103:0/1550969313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4105570 msgr2=0x7ff2a419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:20.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff2a1e00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4105570 0x7ff2a419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff2a1e00640 1 -- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4105ef0 msgr2=0x7ff2a41a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:20.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff2a1e00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4105ef0 0x7ff2a41a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff2a1e00640 1 -- 172.21.15.103:0/1550969313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a41a1590 con 0x7ff2a4101220 2024-09-11T00:22:20.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff2a2800640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4105ef0 0x7ff2a41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:20.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.274+0000 7ff2a1e00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4101220 0x7ff2a419b4e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff29402aa00 tx=0x7ff294003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:20.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.275+0000 7ff29b400640 1 -- 172.21.15.103:0/1550969313 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29403f070 con 0x7ff2a4101220 2024-09-11T00:22:20.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.275+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a41a1820 con 0x7ff2a4101220 2024-09-11T00:22:20.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.275+0000 7ff29b400640 1 -- 172.21.15.103:0/1550969313 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff294031b50 con 0x7ff2a4101220 2024-09-11T00:22:20.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.275+0000 7ff29b400640 1 -- 172.21.15.103:0/1550969313 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294035030 con 0x7ff2a4101220 2024-09-11T00:22:20.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.275+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a41a7f40 con 0x7ff2a4101220 2024-09-11T00:22:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.276+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a4101600 con 0x7ff2a4101220 2024-09-11T00:22:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.278+0000 7ff29b400640 1 -- 172.21.15.103:0/1550969313 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff2940316a0 con 0x7ff2a4101220 2024-09-11T00:22:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.280+0000 7ff29b400640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff2880b53f0 0x7ff2880b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.280+0000 7ff29b400640 1 -- 172.21.15.103:0/1550969313 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff294106a80 con 0x7ff2a4101220 2024-09-11T00:22:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.280+0000 7ff299400640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff2880b53f0 0x7ff2880b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:20.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.281+0000 7ff299400640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff2880b53f0 0x7ff2880b78b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4104e30 tx=0x7ff28c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:20.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.281+0000 7ff29b400640 1 -- 172.21.15.103:0/1550969313 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff2940ccdd0 con 0x7ff2a4101220 2024-09-11T00:22:20.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.473+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff2a41a82b0 con 0x7ff2a4101220 2024-09-11T00:22:20.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.473+0000 7ff29b400640 1 -- 172.21.15.103:0/1550969313 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff2940d1800 con 0x7ff2a4101220 2024-09-11T00:22:20.476 INFO:teuthology.orchestra.run.smithi103.stdout:201863462970 2024-09-11T00:22:20.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff2880b53f0 msgr2=0x7ff2880b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:20.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff2880b53f0 0x7ff2880b78b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4104e30 tx=0x7ff28c005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:20.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4101220 msgr2=0x7ff2a419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:20.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4101220 0x7ff2a419b4e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff29402aa00 tx=0x7ff294003b90 comp rx=0 tx=0).stop 2024-09-11T00:22:20.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 shutdown_connections 2024-09-11T00:22:20.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff2880b53f0 0x7ff2880b78b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2a4105ef0 0x7ff2a41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2a4105570 0x7ff2a419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 --2- 172.21.15.103:0/1550969313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2a4101220 0x7ff2a419b4e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:20.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.475+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 >> 172.21.15.103:0/1550969313 conn(0x7ff2a4078100 msgr2=0x7ff2a4107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:20.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.476+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 shutdown_connections 2024-09-11T00:22:20.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:20.476+0000 7ff2a8c00640 1 -- 172.21.15.103:0/1550969313 wait complete. 2024-09-11T00:22:20.543 INFO:teuthology.orchestra.run.smithi103.stdout:154618822725 2024-09-11T00:22:20.543 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T00:22:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:20 smithi183 ceph-mon[35934]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:20.813 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:20 smithi103 ceph-mon[30578]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:20.819 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:20.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:20 smithi117 ceph-mon[35525]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:21.509 INFO:teuthology.orchestra.run.smithi103.stdout:274877906983 2024-09-11T00:22:21.588 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T00:22:21.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:21 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1550969313' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:22:21.823 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:21 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1550969313' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:22:21.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:21 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1550969313' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.183+0000 7f470d200640 1 -- 172.21.15.103:0/82655618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 msgr2=0x7f470810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.183+0000 7f470d200640 1 --2- 172.21.15.103:0/82655618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470810ac60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009860 tx=0x7f46f402a340 comp rx=0 tx=0).stop 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.183+0000 7f470d200640 1 -- 172.21.15.103:0/82655618 shutdown_connections 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.183+0000 7f470d200640 1 --2- 172.21.15.103:0/82655618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f470810b1a0 0x7f47081119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.183+0000 7f470d200640 1 --2- 172.21.15.103:0/82655618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470810ac60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.183+0000 7f470d200640 1 --2- 172.21.15.103:0/82655618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4708104e10 0x7f47081051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.183+0000 7f470d200640 1 -- 172.21.15.103:0/82655618 >> 172.21.15.103:0/82655618 conn(0x7f4708100840 msgr2=0x7f4708102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.184+0000 7f470d200640 1 -- 172.21.15.103:0/82655618 shutdown_connections 2024-09-11T00:22:22.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.184+0000 7f470d200640 1 -- 172.21.15.103:0/82655618 wait complete. 2024-09-11T00:22:22.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 Processor -- start 2024-09-11T00:22:22.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 -- start start 2024-09-11T00:22:22.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4708104e10 0x7f470819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:22.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:22.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470810b1a0 0x7f47081a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:22.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4708114e60 con 0x7f4708104e10 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4708114ce0 con 0x7f470810a820 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.185+0000 7f470d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4708114fe0 con 0x7f470810b1a0 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4707400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4708104e10 0x7f470819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46650/0 (socket says 172.21.15.103:46650) 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 -- 172.21.15.103:0/770751098 learned_addr learned my addr 172.21.15.103:0/770751098 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4707400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4708104e10 0x7f470819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59686/0 (socket says 172.21.15.103:59686) 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4707e00640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470810b1a0 0x7f47081a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 -- 172.21.15.103:0/770751098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470810b1a0 msgr2=0x7f47081a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470810b1a0 0x7f47081a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 -- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4708104e10 msgr2=0x7f470819b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:22.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4708104e10 0x7f470819b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4706a00640 1 -- 172.21.15.103:0/770751098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47081a12e0 con 0x7f470810a820 2024-09-11T00:22:22.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4707e00640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470810b1a0 0x7f47081a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:22.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.186+0000 7f4707400640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4708104e10 0x7f470819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:22.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.187+0000 7f4706a00640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470819b770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46f4035040 tx=0x7f46f4032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:22.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.187+0000 7f46ffe00640 1 -- 172.21.15.103:0/770751098 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4037040 con 0x7f470810a820 2024-09-11T00:22:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.187+0000 7f46ffe00640 1 -- 172.21.15.103:0/770751098 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f4041070 con 0x7f470810a820 2024-09-11T00:22:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.187+0000 7f46ffe00640 1 -- 172.21.15.103:0/770751098 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4003850 con 0x7f470810a820 2024-09-11T00:22:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.187+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47081a1570 con 0x7f470810a820 2024-09-11T00:22:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.187+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47081a7c90 con 0x7f470810a820 2024-09-11T00:22:22.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.188+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47081060c0 con 0x7f470810a820 2024-09-11T00:22:22.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.191+0000 7f46ffe00640 1 -- 172.21.15.103:0/770751098 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f46f40039f0 con 0x7f470810a820 2024-09-11T00:22:22.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.193+0000 7f46ffe00640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46ec0b5490 0x7f46ec0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.193+0000 7f46ffe00640 1 -- 172.21.15.103:0/770751098 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f46f41064a0 con 0x7f470810a820 2024-09-11T00:22:22.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.193+0000 7f4707400640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46ec0b5490 0x7f46ec0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:22.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.194+0000 7f46ffe00640 1 -- 172.21.15.103:0/770751098 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f46f40ccec0 con 0x7f470810a820 2024-09-11T00:22:22.196 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.194+0000 7f4707400640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46ec0b5490 0x7f46ec0b7950 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009fc0 tx=0x7f46f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:22.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.382+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f47081a7ef0 con 0x7f470810a820 2024-09-11T00:22:22.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.383+0000 7f46ffe00640 1 -- 172.21.15.103:0/770751098 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f46f40cc7c0 con 0x7f470810a820 2024-09-11T00:22:22.385 INFO:teuthology.orchestra.run.smithi103.stdout:180388626495 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46ec0b5490 msgr2=0x7f46ec0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46ec0b5490 0x7f46ec0b7950 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009fc0 tx=0x7f46f8005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 msgr2=0x7f470819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470819b770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f46f4035040 tx=0x7f46f4032d50 comp rx=0 tx=0).stop 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 shutdown_connections 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46ec0b5490 0x7f46ec0b7950 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f470810b1a0 0x7f47081a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f470810a820 0x7f470819b770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 --2- 172.21.15.103:0/770751098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4708104e10 0x7f470819b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 >> 172.21.15.103:0/770751098 conn(0x7f4708100840 msgr2=0x7f4708102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 shutdown_connections 2024-09-11T00:22:22.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:22.386+0000 7f470d200640 1 -- 172.21.15.103:0/770751098 wait complete. 2024-09-11T00:22:22.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:22 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/770751098' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:22:22.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:22 smithi103 ceph-mon[30578]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:22 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/770751098' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:22:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:22 smithi183 ceph-mon[35934]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:22 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/770751098' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-11T00:22:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:22 smithi117 ceph-mon[35525]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:23.133 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462967 got 201863462970 for osd.7 2024-09-11T00:22:23.133 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:23.510 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182478 got 107374182480 for osd.3 2024-09-11T00:22:23.510 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:24.020 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:24.025 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:24.027 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:24.031 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:24.073 INFO:teuthology.orchestra.run.smithi103.stdout:253403070511 2024-09-11T00:22:24.074 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T00:22:24.482 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626492 got 180388626495 for osd.6 2024-09-11T00:22:24.482 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:24.540 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:24.778 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:24 smithi103 ceph-mon[30578]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:24 smithi183 ceph-mon[35934]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:24 smithi117 ceph-mon[35525]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:25.723 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:25 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:22:25.758 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:25 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:22:25.798 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:25 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:22:26.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 -- 172.21.15.103:0/3308858913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 msgr2=0x7fc390108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:26.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 --2- 172.21.15.103:0/3308858913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 0x7fc390108c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009860 tx=0x7fc37c02a340 comp rx=0 tx=0).stop 2024-09-11T00:22:26.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 -- 172.21.15.103:0/3308858913 shutdown_connections 2024-09-11T00:22:26.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 --2- 172.21.15.103:0/3308858913 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390109150 0x7fc39010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 --2- 172.21.15.103:0/3308858913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 0x7fc390108c10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 --2- 172.21.15.103:0/3308858913 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390102dc0 0x7fc3901031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 -- 172.21.15.103:0/3308858913 >> 172.21.15.103:0/3308858913 conn(0x7fc3900fe830 msgr2=0x7fc390100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:26.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 -- 172.21.15.103:0/3308858913 shutdown_connections 2024-09-11T00:22:26.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.056+0000 7fc397c00640 1 -- 172.21.15.103:0/3308858913 wait complete. 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 Processor -- start 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 -- start start 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390102dc0 0x7fc390076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 0x7fc3900754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390109150 0x7fc390075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc390112dd0 con 0x7fc390102dc0 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc390112c50 con 0x7fc390109150 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.057+0000 7fc397c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc390112f50 con 0x7fc3901087d0 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390109150 0x7fc390075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:26.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390109150 0x7fc390075ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46694/0 (socket says 172.21.15.103:46694) 2024-09-11T00:22:26.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 -- 172.21.15.103:0/1693197338 learned_addr learned my addr 172.21.15.103:0/1693197338 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:26.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395200640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390102dc0 0x7fc390076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:26.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc38fe00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 0x7fc3900754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 -- 172.21.15.103:0/1693197338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 msgr2=0x7fc3900754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 0x7fc3900754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 -- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390102dc0 msgr2=0x7fc390076f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390102dc0 0x7fc390076f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.058+0000 7fc395c00640 1 -- 172.21.15.103:0/1693197338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3901abc40 con 0x7fc390109150 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.059+0000 7fc395c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390109150 0x7fc390075ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc38800eb10 tx=0x7fc38800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.059+0000 7fc38d600640 1 -- 172.21.15.103:0/1693197338 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38800ce20 con 0x7fc390109150 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.059+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3901abe10 con 0x7fc390109150 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.059+0000 7fc38d600640 1 -- 172.21.15.103:0/1693197338 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc388005020 con 0x7fc390109150 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.059+0000 7fc38d600640 1 -- 172.21.15.103:0/1693197338 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388010690 con 0x7fc390109150 2024-09-11T00:22:26.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.059+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901ac2c0 con 0x7fc390109150 2024-09-11T00:22:26.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.061+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3901031a0 con 0x7fc390109150 2024-09-11T00:22:26.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.063+0000 7fc38d600640 1 -- 172.21.15.103:0/1693197338 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fc388005510 con 0x7fc390109150 2024-09-11T00:22:26.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.064+0000 7fc38d600640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc3740b53f0 0x7fc3740b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:26.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.064+0000 7fc38d600640 1 -- 172.21.15.103:0/1693197338 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fc3880e6680 con 0x7fc390109150 2024-09-11T00:22:26.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.065+0000 7fc395200640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc3740b53f0 0x7fc3740b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:26.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.065+0000 7fc395200640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc3740b53f0 0x7fc3740b78b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc380005fd0 tx=0x7fc380009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:26.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.068+0000 7fc38d600640 1 -- 172.21.15.103:0/1693197338 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc3880ad020 con 0x7fc390109150 2024-09-11T00:22:26.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.255+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc39002e5c0 con 0x7fc390109150 2024-09-11T00:22:26.257 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.255+0000 7fc38d600640 1 -- 172.21.15.103:0/1693197338 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3880ac920 con 0x7fc390109150 2024-09-11T00:22:26.258 INFO:teuthology.orchestra.run.smithi103.stdout:77309411415 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc3740b53f0 msgr2=0x7fc3740b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc3740b53f0 0x7fc3740b78b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc380005fd0 tx=0x7fc380009260 comp rx=0 tx=0).stop 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390109150 msgr2=0x7fc390075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390109150 0x7fc390075ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc38800eb10 tx=0x7fc38800ef10 comp rx=0 tx=0).stop 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 shutdown_connections 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc3740b53f0 0x7fc3740b78b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390109150 0x7fc390075ad0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3901087d0 0x7fc3900754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 --2- 172.21.15.103:0/1693197338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc390102dc0 0x7fc390076f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:26.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 >> 172.21.15.103:0/1693197338 conn(0x7fc3900fe830 msgr2=0x7fc390100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:26.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 shutdown_connections 2024-09-11T00:22:26.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:26.258+0000 7fc397c00640 1 -- 172.21.15.103:0/1693197338 wait complete. 2024-09-11T00:22:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:26 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/522237878' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:22:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:26 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1693197338' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:22:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:26 smithi103 ceph-mon[30578]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:26 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:26 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:26 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:26 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/522237878' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:26 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1693197338' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:26 smithi117 ceph-mon[35525]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:26 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:26 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:26 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:26 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:26 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/522237878' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-11T00:22:27.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:26 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1693197338' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-11T00:22:27.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:26 smithi183 ceph-mon[35934]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:27.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:26 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:26 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.440+0000 7faa56c00640 1 -- 172.21.15.103:0/2441389064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa50109150 msgr2=0x7faa5010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.440+0000 7faa56c00640 1 --2- 172.21.15.103:0/2441389064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa50109150 0x7faa5010f950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa48009900 tx=0x7faa4802a410 comp rx=0 tx=0).stop 2024-09-11T00:22:27.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.440+0000 7faa56c00640 1 -- 172.21.15.103:0/2441389064 shutdown_connections 2024-09-11T00:22:27.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.440+0000 7faa56c00640 1 --2- 172.21.15.103:0/2441389064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa50109150 0x7faa5010f950 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.440+0000 7faa56c00640 1 --2- 172.21.15.103:0/2441389064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa501087d0 0x7faa50108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.440+0000 7faa56c00640 1 --2- 172.21.15.103:0/2441389064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa50102dc0 0x7faa501031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.440+0000 7faa56c00640 1 -- 172.21.15.103:0/2441389064 >> 172.21.15.103:0/2441389064 conn(0x7faa500fe830 msgr2=0x7faa50100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:27.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.441+0000 7faa56c00640 1 -- 172.21.15.103:0/2441389064 shutdown_connections 2024-09-11T00:22:27.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.441+0000 7faa56c00640 1 -- 172.21.15.103:0/2441389064 wait complete. 2024-09-11T00:22:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.441+0000 7faa56c00640 1 Processor -- start 2024-09-11T00:22:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa56c00640 1 -- start start 2024-09-11T00:22:27.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa56c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50102dc0 0x7faa501992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa56c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa501087d0 0x7faa50199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa56c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa50109150 0x7faa5019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa56c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa50112e20 con 0x7faa501087d0 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa56c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7faa50112ca0 con 0x7faa50102dc0 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa56c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faa50112fa0 con 0x7faa50109150 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa4fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50102dc0 0x7faa501992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa4fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50102dc0 0x7faa501992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46714/0 (socket says 172.21.15.103:46714) 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa4fe00640 1 -- 172.21.15.103:0/2089653565 learned_addr learned my addr 172.21.15.103:0/2089653565 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa54c00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa50109150 0x7faa5019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.442+0000 7faa4f400640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa501087d0 0x7faa50199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4fe00640 1 -- 172.21.15.103:0/2089653565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa50109150 msgr2=0x7faa5019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4fe00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa50109150 0x7faa5019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4fe00640 1 -- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa501087d0 msgr2=0x7faa50199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4fe00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa501087d0 0x7faa50199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4fe00640 1 -- 172.21.15.103:0/2089653565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5019f3a0 con 0x7faa50102dc0 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4fe00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50102dc0 0x7faa501992f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ebc0 tx=0x7faa4000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4cc00640 1 -- 172.21.15.103:0/2089653565 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40010630 con 0x7faa50102dc0 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4cc00640 1 -- 172.21.15.103:0/2089653565 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa40010c80 con 0x7faa50102dc0 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5019f690 con 0x7faa50102dc0 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.443+0000 7faa4cc00640 1 -- 172.21.15.103:0/2089653565 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40018b60 con 0x7faa50102dc0 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.444+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa501a5d50 con 0x7faa50102dc0 2024-09-11T00:22:27.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.445+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa501031a0 con 0x7faa50102dc0 2024-09-11T00:22:27.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.447+0000 7faa4cc00640 1 -- 172.21.15.103:0/2089653565 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7faa400107d0 con 0x7faa50102dc0 2024-09-11T00:22:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.448+0000 7faa4cc00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7faa340b5370 0x7faa340b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.448+0000 7faa4cc00640 1 -- 172.21.15.103:0/2089653565 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7faa400e66f0 con 0x7faa50102dc0 2024-09-11T00:22:27.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.449+0000 7faa4f400640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7faa340b5370 0x7faa340b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.449+0000 7faa4f400640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7faa340b5370 0x7faa340b7830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa50104ea0 tx=0x7faa38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:27.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.452+0000 7faa4cc00640 1 -- 172.21.15.103:0/2089653565 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7faa400aca40 con 0x7faa50102dc0 2024-09-11T00:22:27.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.644+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7faa5002e5c0 con 0x7faa50102dc0 2024-09-11T00:22:27.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.645+0000 7faa4cc00640 1 -- 172.21.15.103:0/2089653565 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7faa400b1470 con 0x7faa50102dc0 2024-09-11T00:22:27.647 INFO:teuthology.orchestra.run.smithi103.stdout:300647710758 2024-09-11T00:22:27.648 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411411 got 77309411415 for osd.2 2024-09-11T00:22:27.648 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.647+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7faa340b5370 msgr2=0x7faa340b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.647+0000 7faa56c00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7faa340b5370 0x7faa340b7830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7faa50104ea0 tx=0x7faa38005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.647+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50102dc0 msgr2=0x7faa501992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.647+0000 7faa56c00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50102dc0 0x7faa501992f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ebc0 tx=0x7faa4000c6a0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 shutdown_connections 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7faa340b5370 0x7faa340b7830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faa50109150 0x7faa5019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa501087d0 0x7faa50199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 --2- 172.21.15.103:0/2089653565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa50102dc0 0x7faa501992f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 >> 172.21.15.103:0/2089653565 conn(0x7faa500fe830 msgr2=0x7faa500ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 shutdown_connections 2024-09-11T00:22:27.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.648+0000 7faa56c00640 1 -- 172.21.15.103:0/2089653565 wait complete. 2024-09-11T00:22:27.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-11T00:22:27.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:27 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]} : dispatch 2024-09-11T00:22:27.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:27 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2089653565' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:22:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-11T00:22:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:27 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]} : dispatch 2024-09-11T00:22:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:27 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2089653565' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.924+0000 7fe043200640 1 -- 172.21.15.103:0/1866906853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 msgr2=0x7fe044111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.924+0000 7fe043200640 1 --2- 172.21.15.103:0/1866906853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 0x7fe044111610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009ca0 tx=0x7fe03c02a8b0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.924+0000 7fe043200640 1 -- 172.21.15.103:0/1866906853 shutdown_connections 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.924+0000 7fe043200640 1 --2- 172.21.15.103:0/1866906853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 0x7fe044111610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.924+0000 7fe043200640 1 --2- 172.21.15.103:0/1866906853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe044075de0 0x7fe04410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.924+0000 7fe043200640 1 --2- 172.21.15.103:0/1866906853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0440754c0 0x7fe0440758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.924+0000 7fe043200640 1 -- 172.21.15.103:0/1866906853 >> 172.21.15.103:0/1866906853 conn(0x7fe0440fe660 msgr2=0x7fe044100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.925+0000 7fe043200640 1 -- 172.21.15.103:0/1866906853 shutdown_connections 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.925+0000 7fe043200640 1 -- 172.21.15.103:0/1866906853 wait complete. 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.925+0000 7fe043200640 1 Processor -- start 2024-09-11T00:22:27.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe043200640 1 -- start start 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe043200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0440754c0 0x7fe04419b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe043200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe044075de0 0x7fe04419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe043200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 0x7fe0441a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe043200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe044114e50 con 0x7fe0440754c0 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe043200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe044114cd0 con 0x7fe044075de0 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe043200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe044114fd0 con 0x7fe04410f220 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe041e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0440754c0 0x7fe04419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe041e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0440754c0 0x7fe04419b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59754/0 (socket says 172.21.15.103:59754) 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe041e00640 1 -- 172.21.15.103:0/1364830230 learned_addr learned my addr 172.21.15.103:0/1364830230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe041400640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe044075de0 0x7fe04419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.926+0000 7fe042800640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 0x7fe0441a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe041400640 1 -- 172.21.15.103:0/1364830230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 msgr2=0x7fe0441a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe041400640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 0x7fe0441a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe041400640 1 -- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0440754c0 msgr2=0x7fe04419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe041400640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0440754c0 0x7fe04419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe041400640 1 -- 172.21.15.103:0/1364830230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0441a12f0 con 0x7fe044075de0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe041e00640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0440754c0 0x7fe04419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe041400640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe044075de0 0x7fe04419b780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe03000e9f0 tx=0x7fe03000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe03aa00640 1 -- 172.21.15.103:0/1364830230 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03000ccc0 con 0x7fe044075de0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe03aa00640 1 -- 172.21.15.103:0/1364830230 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0300108e0 con 0x7fe044075de0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe03aa00640 1 -- 172.21.15.103:0/1364830230 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe030018880 con 0x7fe044075de0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.927+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0441a15e0 con 0x7fe044075de0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.928+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0441a7ca0 con 0x7fe044075de0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.929+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0440758a0 con 0x7fe044075de0 2024-09-11T00:22:27.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.931+0000 7fe03aa00640 1 -- 172.21.15.103:0/1364830230 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fe030010430 con 0x7fe044075de0 2024-09-11T00:22:27.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.932+0000 7fe03aa00640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe0280b5490 0x7fe0280b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:27.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.932+0000 7fe03aa00640 1 -- 172.21.15.103:0/1364830230 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9150+0+0 (secure 0 0 0) 0x7fe0300e63f0 con 0x7fe044075de0 2024-09-11T00:22:27.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.932+0000 7fe041e00640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe0280b5490 0x7fe0280b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:27.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.933+0000 7fe041e00640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe0280b5490 0x7fe0280b7950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe034011550 tx=0x7fe034005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:27.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:27.933+0000 7fe03aa00640 1 -- 172.21.15.103:0/1364830230 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe0300ac710 con 0x7fe044075de0 2024-09-11T00:22:28.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:28.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:28.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]} : dispatch 2024-09-11T00:22:28.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:27 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]} : dispatch 2024-09-11T00:22:28.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:27 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2089653565' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-11T00:22:28.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.125+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe04402e5c0 con 0x7fe044075de0 2024-09-11T00:22:28.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.126+0000 7fe03aa00640 1 -- 172.21.15.103:0/1364830230 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe0300b1140 con 0x7fe044075de0 2024-09-11T00:22:28.128 INFO:teuthology.orchestra.run.smithi103.stdout:227633266742 2024-09-11T00:22:28.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe0280b5490 msgr2=0x7fe0280b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe0280b5490 0x7fe0280b7950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe034011550 tx=0x7fe034005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe044075de0 msgr2=0x7fe04419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe044075de0 0x7fe04419b780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe03000e9f0 tx=0x7fe03000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 shutdown_connections 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe0280b5490 0x7fe0280b7950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe04410f220 0x7fe0441a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe044075de0 0x7fe04419b780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 --2- 172.21.15.103:0/1364830230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe0440754c0 0x7fe04419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.128+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 >> 172.21.15.103:0/1364830230 conn(0x7fe0440fe660 msgr2=0x7fe044106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.129+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 shutdown_connections 2024-09-11T00:22:28.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:28.129+0000 7fe043200640 1 -- 172.21.15.103:0/1364830230 wait complete. 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 -- 172.21.15.103:0/1988536905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 msgr2=0x7f4e8c111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 --2- 172.21.15.103:0/1988536905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 0x7f4e8c111b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4e84009900 tx=0x7f4e8402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 -- 172.21.15.103:0/1988536905 shutdown_connections 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 --2- 172.21.15.103:0/1988536905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 0x7f4e8c111b50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 --2- 172.21.15.103:0/1988536905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c10a9d0 0x7f4e8c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 --2- 172.21.15.103:0/1988536905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c104fc0 0x7f4e8c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 -- 172.21.15.103:0/1988536905 >> 172.21.15.103:0/1988536905 conn(0x7f4e8c1009f0 msgr2=0x7f4e8c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 -- 172.21.15.103:0/1988536905 shutdown_connections 2024-09-11T00:22:28.626 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.478+0000 7f4e91c00640 1 -- 172.21.15.103:0/1988536905 wait complete. 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 Processor -- start 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 -- start start 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c104fc0 0x7f4e8c19b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c10a9d0 0x7f4e8c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 0x7f4e8c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c114f80 con 0x7f4e8c104fc0 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c114e00 con 0x7f4e8c10b350 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.479+0000 7f4e91c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c115100 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c104fc0 0x7f4e8c19b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c10a9d0 0x7f4e8c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c10a9d0 0x7f4e8c19b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:36718/0 (socket says 172.21.15.103:36718) 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 -- 172.21.15.103:0/522237878 learned_addr learned my addr 172.21.15.103:0/522237878 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:28.627 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c104fc0 0x7f4e8c19b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59696/0 (socket says 172.21.15.103:59696) 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8b400640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 0x7f4e8c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 -- 172.21.15.103:0/522237878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 msgr2=0x7f4e8c1a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 0x7f4e8c1a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 -- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c104fc0 msgr2=0x7f4e8c19b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c104fc0 0x7f4e8c19b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 -- 172.21.15.103:0/522237878 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c1a1560 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8aa00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c104fc0 0x7f4e8c19b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.480+0000 7f4e8a000640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c10a9d0 0x7f4e8c19b9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800ea60 tx=0x7f4e7800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.481+0000 7f4e83400640 1 -- 172.21.15.103:0/522237878 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800cde0 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.481+0000 7f4e83400640 1 -- 172.21.15.103:0/522237878 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e780035f0 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.481+0000 7f4e83400640 1 -- 172.21.15.103:0/522237878 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78010690 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.481+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c1a1850 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.481+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c1a7f10 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.482+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8c1053a0 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.484+0000 7f4e83400640 1 -- 172.21.15.103:0/522237878 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f4e78004000 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.485+0000 7f4e83400640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b53f0 0x7f4e700b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.486+0000 7f4e83400640 1 -- 172.21.15.103:0/522237878 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f4e780e6310 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.486+0000 7f4e8aa00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b53f0 0x7f4e700b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.486+0000 7f4e8aa00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b53f0 0x7f4e700b78b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c005fd0 tx=0x7f4e7c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.489+0000 7f4e83400640 1 -- 172.21.15.103:0/522237878 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4e780accb0 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.733+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4e8c02e5c0 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.733+0000 7f4e83400640 1 -- 172.21.15.103:0/522237878 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e780ac5b0 con 0x7f4e8c10a9d0 2024-09-11T00:22:28.629 INFO:teuthology.orchestra.run.smithi103.stdout:34359738466 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.735+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b53f0 msgr2=0x7f4e700b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.735+0000 7f4e91c00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b53f0 0x7f4e700b78b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c005fd0 tx=0x7f4e7c009500 comp rx=0 tx=0).stop 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.735+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c10a9d0 msgr2=0x7f4e8c19b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.735+0000 7f4e91c00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c10a9d0 0x7f4e8c19b9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800ea60 tx=0x7f4e7800ef20 comp rx=0 tx=0).stop 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 shutdown_connections 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b53f0 0x7f4e700b78b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c10b350 0x7f4e8c1a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e8c10a9d0 0x7f4e8c19b9f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 --2- 172.21.15.103:0/522237878 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e8c104fc0 0x7f4e8c19b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 >> 172.21.15.103:0/522237878 conn(0x7f4e8c1009f0 msgr2=0x7f4e8c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 shutdown_connections 2024-09-11T00:22:28.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:25.736+0000 7f4e91c00640 1 -- 172.21.15.103:0/522237878 wait complete. 2024-09-11T00:22:28.833 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:28.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-11T00:22:28.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:28 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-11T00:22:28.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:28 smithi103 ceph-mon[30578]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T00:22:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:28 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1364830230' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:22:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:28 smithi103 ceph-mon[30578]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T00:22:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-11T00:22:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:28 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-11T00:22:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:28 smithi117 ceph-mon[35525]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T00:22:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:28 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1364830230' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:22:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:28 smithi117 ceph-mon[35525]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T00:22:29.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-11T00:22:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:28 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-11T00:22:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:28 smithi183 ceph-mon[35934]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T00:22:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:28 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1364830230' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-11T00:22:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:28 smithi183 ceph-mon[35934]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T00:22:29.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 -- 172.21.15.103:0/3161313626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa37010b1a0 msgr2=0x7fa3701119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:29.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 --2- 172.21.15.103:0/3161313626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa37010b1a0 0x7fa3701119a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa368009ca0 tx=0x7fa36802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 -- 172.21.15.103:0/3161313626 shutdown_connections 2024-09-11T00:22:29.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 --2- 172.21.15.103:0/3161313626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa37010b1a0 0x7fa3701119a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 --2- 172.21.15.103:0/3161313626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37010a820 0x7fa37010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 --2- 172.21.15.103:0/3161313626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa370104e10 0x7fa3701051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 -- 172.21.15.103:0/3161313626 >> 172.21.15.103:0/3161313626 conn(0x7fa370100840 msgr2=0x7fa370102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:29.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 -- 172.21.15.103:0/3161313626 shutdown_connections 2024-09-11T00:22:29.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.075+0000 7fa376a00640 1 -- 172.21.15.103:0/3161313626 wait complete. 2024-09-11T00:22:29.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.076+0000 7fa376a00640 1 Processor -- start 2024-09-11T00:22:29.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.076+0000 7fa376a00640 1 -- start start 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa376a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa370104e10 0x7fa37019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa376a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37010a820 0x7fa37019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa376a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa37010b1a0 0x7fa3701a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa376a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa370114e20 con 0x7fa370104e10 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa376a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa370114ca0 con 0x7fa37010a820 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa376a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa370114fa0 con 0x7fa37010b1a0 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa375600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa370104e10 0x7fa37019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:29.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa375600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa370104e10 0x7fa37019b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59776/0 (socket says 172.21.15.103:59776) 2024-09-11T00:22:29.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa375600640 1 -- 172.21.15.103:0/3053868058 learned_addr learned my addr 172.21.15.103:0/3053868058 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:29.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa376000640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa37010b1a0 0x7fa3701a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:29.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.077+0000 7fa374c00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37010a820 0x7fa37019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:29.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa375600640 1 -- 172.21.15.103:0/3053868058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa37010b1a0 msgr2=0x7fa3701a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:29.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa375600640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa37010b1a0 0x7fa3701a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa375600640 1 -- 172.21.15.103:0/3053868058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37010a820 msgr2=0x7fa37019b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa375600640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37010a820 0x7fa37019b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa375600640 1 -- 172.21.15.103:0/3053868058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3701a12e0 con 0x7fa370104e10 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa375600640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa370104e10 0x7fa37019b230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e9f0 tx=0x7fa36400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa36e000640 1 -- 172.21.15.103:0/3053868058 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36400cd80 con 0x7fa370104e10 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa36e000640 1 -- 172.21.15.103:0/3053868058 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3640054d0 con 0x7fa370104e10 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa36e000640 1 -- 172.21.15.103:0/3053868058 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364010690 con 0x7fa370104e10 2024-09-11T00:22:29.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3701a15d0 con 0x7fa370104e10 2024-09-11T00:22:29.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.078+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3701a7c90 con 0x7fa370104e10 2024-09-11T00:22:29.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.079+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3701051f0 con 0x7fa370104e10 2024-09-11T00:22:29.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.082+0000 7fa36e000640 1 -- 172.21.15.103:0/3053868058 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fa364004e30 con 0x7fa370104e10 2024-09-11T00:22:29.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.084+0000 7fa36e000640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3580b5490 0x7fa3580b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:29.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.084+0000 7fa36e000640 1 -- 172.21.15.103:0/3053868058 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa3640e66f0 con 0x7fa370104e10 2024-09-11T00:22:29.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.084+0000 7fa374c00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3580b5490 0x7fa3580b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:29.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.085+0000 7fa374c00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3580b5490 0x7fa3580b7950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa370103230 tx=0x7fa35c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:29.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.086+0000 7fa36e000640 1 -- 172.21.15.103:0/3053868058 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa364056220 con 0x7fa370104e10 2024-09-11T00:22:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.277+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa3701a8060 con 0x7fa370104e10 2024-09-11T00:22:29.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.278+0000 7fa36e000640 1 -- 172.21.15.103:0/3053868058 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa3640ad060 con 0x7fa370104e10 2024-09-11T00:22:29.280 INFO:teuthology.orchestra.run.smithi103.stdout:154618822727 2024-09-11T00:22:29.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3580b5490 msgr2=0x7fa3580b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:29.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3580b5490 0x7fa3580b7950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa370103230 tx=0x7fa35c005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:29.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa370104e10 msgr2=0x7fa37019b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:29.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa370104e10 0x7fa37019b230 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa36400e9f0 tx=0x7fa36400edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 shutdown_connections 2024-09-11T00:22:29.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3580b5490 0x7fa3580b7950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa37010b1a0 0x7fa3701a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa37010a820 0x7fa37019b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 --2- 172.21.15.103:0/3053868058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa370104e10 0x7fa37019b230 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:29.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 >> 172.21.15.103:0/3053868058 conn(0x7fa370100840 msgr2=0x7fa370100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:29.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 shutdown_connections 2024-09-11T00:22:29.283 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:29.280+0000 7fa376a00640 1 -- 172.21.15.103:0/3053868058 wait complete. 2024-09-11T00:22:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:29 smithi117 ceph-mon[35525]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T00:22:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:29 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3053868058' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:22:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:29 smithi183 ceph-mon[35934]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T00:22:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:29 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3053868058' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:22:30.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:29 smithi103 ceph-mon[30578]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T00:22:30.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:29 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3053868058' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-11T00:22:30.293 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266740 got 227633266742 for osd.8 2024-09-11T00:22:30.294 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:30.810 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:30 smithi103 ceph-mon[30578]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T00:22:30.821 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:30.852 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822725 got 154618822727 for osd.5 2024-09-11T00:22:30.852 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:30 smithi117 ceph-mon[35525]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T00:22:31.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:30 smithi183 ceph-mon[35934]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T00:22:31.384 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738464 got 34359738466 for osd.0 2024-09-11T00:22:31.384 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 -- 172.21.15.103:0/4294385551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06e8108620 msgr2=0x7f06e8108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 --2- 172.21.15.103:0/4294385551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06e8108620 0x7f06e8108a60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009900 tx=0x7f06d402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 -- 172.21.15.103:0/4294385551 shutdown_connections 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 --2- 172.21.15.103:0/4294385551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8108fa0 0x7f06e810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 --2- 172.21.15.103:0/4294385551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06e8108620 0x7f06e8108a60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 --2- 172.21.15.103:0/4294385551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8102c10 0x7f06e8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 -- 172.21.15.103:0/4294385551 >> 172.21.15.103:0/4294385551 conn(0x7f06e80fe660 msgr2=0x7f06e8100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.617+0000 7f06eda00640 1 -- 172.21.15.103:0/4294385551 shutdown_connections 2024-09-11T00:22:31.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.618+0000 7f06eda00640 1 -- 172.21.15.103:0/4294385551 wait complete. 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.618+0000 7f06eda00640 1 Processor -- start 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06eda00640 1 -- start start 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06eda00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06e8102c10 0x7f06e819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06eda00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8108620 0x7f06e819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06eda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8108fa0 0x7f06e81a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06eda00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8112c60 con 0x7f06e8108fa0 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06eda00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8112ae0 con 0x7f06e8102c10 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06eda00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06e8112de0 con 0x7f06e8108620 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06ed000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8108fa0 0x7f06e81a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06e7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8108620 0x7f06e819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06e7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8108620 0x7f06e819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:36818/0 (socket says 172.21.15.103:36818) 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.619+0000 7f06e7400640 1 -- 172.21.15.103:0/3492311630 learned_addr learned my addr 172.21.15.103:0/3492311630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.620+0000 7f06e7400640 1 -- 172.21.15.103:0/3492311630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06e8102c10 msgr2=0x7f06e819b210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.620+0000 7f06e7400640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06e8102c10 0x7f06e819b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.707 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.620+0000 7f06e7400640 1 -- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8108fa0 msgr2=0x7f06e81a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.620+0000 7f06e7400640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8108fa0 0x7f06e81a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.620+0000 7f06e7400640 1 -- 172.21.15.103:0/3492311630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e81a12c0 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.620+0000 7f06e7400640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8108620 0x7f06e819b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06d4005940 tx=0x7f06d4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.621+0000 7f06e4c00640 1 -- 172.21.15.103:0/3492311630 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d403f030 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.621+0000 7f06e4c00640 1 -- 172.21.15.103:0/3492311630 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d4003aa0 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.621+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e81a1550 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.621+0000 7f06e4c00640 1 -- 172.21.15.103:0/3492311630 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4035070 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.621+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e81a7c70 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.622+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06e8102ff0 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.625+0000 7f06e4c00640 1 -- 172.21.15.103:0/3492311630 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f06d403bc00 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.626+0000 7f06e4c00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f06cc0b5410 0x7f06cc0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.627+0000 7f06e4c00640 1 -- 172.21.15.103:0/3492311630 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f06d4106e00 con 0x7f06e8108620 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.627+0000 7f06e7e00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f06cc0b5410 0x7f06cc0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:31.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.627+0000 7f06e4c00640 1 -- 172.21.15.103:0/3492311630 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f06d40cd770 con 0x7f06e8108620 2024-09-11T00:22:31.709 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.628+0000 7f06e7e00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f06cc0b5410 0x7f06cc0b78d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06d800b990 tx=0x7f06d800b370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:31.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.822+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f06e819c340 con 0x7f06e8108620 2024-09-11T00:22:31.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.823+0000 7f06e4c00640 1 -- 172.21.15.103:0/3492311630 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f06e819c340 con 0x7f06e8108620 2024-09-11T00:22:31.829 INFO:teuthology.orchestra.run.smithi103.stdout:274877906987 2024-09-11T00:22:31.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f06cc0b5410 msgr2=0x7f06cc0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:31.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f06cc0b5410 0x7f06cc0b78d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06d800b990 tx=0x7f06d800b370 comp rx=0 tx=0).stop 2024-09-11T00:22:31.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8108620 msgr2=0x7f06e819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8108620 0x7f06e819b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f06d4005940 tx=0x7f06d4004270 comp rx=0 tx=0).stop 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 shutdown_connections 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f06cc0b5410 0x7f06cc0b78d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06e8108fa0 0x7f06e81a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06e8108620 0x7f06e819b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 --2- 172.21.15.103:0/3492311630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06e8102c10 0x7f06e819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 >> 172.21.15.103:0/3492311630 conn(0x7f06e80fe660 msgr2=0x7f06e8104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 shutdown_connections 2024-09-11T00:22:31.832 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:31.825+0000 7f06eda00640 1 -- 172.21.15.103:0/3492311630 wait complete. 2024-09-11T00:22:31.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:31 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3492311630' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:22:32.164 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710754 got 300647710758 for osd.11 2024-09-11T00:22:32.165 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:32.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:31 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3492311630' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:22:32.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:31 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3492311630' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-11T00:22:33.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.078+0000 7ff1f4800640 1 -- 172.21.15.103:0/3586136753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10b330 msgr2=0x7ff1ec111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.078+0000 7ff1f4800640 1 --2- 172.21.15.103:0/3586136753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10b330 0x7ff1ec111b30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8009ca0 tx=0x7ff1e802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.079+0000 7ff1f4800640 1 -- 172.21.15.103:0/3586136753 shutdown_connections 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.079+0000 7ff1f4800640 1 --2- 172.21.15.103:0/3586136753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10b330 0x7ff1ec111b30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.079+0000 7ff1f4800640 1 --2- 172.21.15.103:0/3586136753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1ec10a9b0 0x7ff1ec10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.079+0000 7ff1f4800640 1 --2- 172.21.15.103:0/3586136753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 0x7ff1ec105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.079+0000 7ff1f4800640 1 -- 172.21.15.103:0/3586136753 >> 172.21.15.103:0/3586136753 conn(0x7ff1ec100a50 msgr2=0x7ff1ec102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.079+0000 7ff1f4800640 1 -- 172.21.15.103:0/3586136753 shutdown_connections 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.079+0000 7ff1f4800640 1 -- 172.21.15.103:0/3586136753 wait complete. 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 Processor -- start 2024-09-11T00:22:33.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 -- start start 2024-09-11T00:22:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 0x7ff1ec19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10a9b0 0x7ff1ec19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1ec10b330 0x7ff1ec1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec114f60 con 0x7ff1ec10a9b0 2024-09-11T00:22:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec114de0 con 0x7ff1ec10b330 2024-09-11T00:22:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f4800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ec1150e0 con 0x7ff1ec104fa0 2024-09-11T00:22:33.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10a9b0 0x7ff1ec19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.080+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 0x7ff1ec19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 0x7ff1ec19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:47536/0 (socket says 172.21.15.103:47536) 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 -- 172.21.15.103:0/2522873772 learned_addr learned my addr 172.21.15.103:0/2522873772 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f2800640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1ec10b330 0x7ff1ec1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 -- 172.21.15.103:0/2522873772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1ec10b330 msgr2=0x7ff1ec1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1ec10b330 0x7ff1ec1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 -- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10a9b0 msgr2=0x7ff1ec19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10a9b0 0x7ff1ec19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 -- 172.21.15.103:0/2522873772 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec1a1500 con 0x7ff1ec104fa0 2024-09-11T00:22:33.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1400640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10a9b0 0x7ff1ec19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:33.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.081+0000 7ff1f1e00640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 0x7ff1ec19b450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e9f0 tx=0x7ff1e000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:33.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.082+0000 7ff1e6a00640 1 -- 172.21.15.103:0/2522873772 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000cd80 con 0x7ff1ec104fa0 2024-09-11T00:22:33.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.082+0000 7ff1e6a00640 1 -- 172.21.15.103:0/2522873772 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e00054d0 con 0x7ff1ec104fa0 2024-09-11T00:22:33.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.082+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ec1a17f0 con 0x7ff1ec104fa0 2024-09-11T00:22:33.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.082+0000 7ff1e6a00640 1 -- 172.21.15.103:0/2522873772 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0010690 con 0x7ff1ec104fa0 2024-09-11T00:22:33.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.082+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ec1a7eb0 con 0x7ff1ec104fa0 2024-09-11T00:22:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.083+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1ec105380 con 0x7ff1ec104fa0 2024-09-11T00:22:33.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.085+0000 7ff1e6a00640 1 -- 172.21.15.103:0/2522873772 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff1e0004e90 con 0x7ff1ec104fa0 2024-09-11T00:22:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.086+0000 7ff1e6a00640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff1d40b53f0 0x7ff1d40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.087+0000 7ff1e6a00640 1 -- 172.21.15.103:0/2522873772 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff1e00e6530 con 0x7ff1ec104fa0 2024-09-11T00:22:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.087+0000 7ff1f1400640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff1d40b53f0 0x7ff1d40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:33.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.087+0000 7ff1f1400640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff1d40b53f0 0x7ff1d40b78b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0095f0 tx=0x7ff1dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:33.092 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.090+0000 7ff1e6a00640 1 -- 172.21.15.103:0/2522873772 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff1e00acea0 con 0x7ff1ec104fa0 2024-09-11T00:22:33.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:32 smithi103 ceph-mon[30578]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:33.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:32 smithi117 ceph-mon[35525]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:33.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:32 smithi183 ceph-mon[35934]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:33.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.279+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff1ec02e5c0 con 0x7ff1ec104fa0 2024-09-11T00:22:33.282 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.280+0000 7ff1e6a00640 1 -- 172.21.15.103:0/2522873772 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff1e00ac7a0 con 0x7ff1ec104fa0 2024-09-11T00:22:33.282 INFO:teuthology.orchestra.run.smithi103.stdout:253403070514 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff1d40b53f0 msgr2=0x7ff1d40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff1d40b53f0 0x7ff1d40b78b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0095f0 tx=0x7ff1dc005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 msgr2=0x7ff1ec19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 0x7ff1ec19b450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e9f0 tx=0x7ff1e000edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 shutdown_connections 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff1d40b53f0 0x7ff1d40b78b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1ec10b330 0x7ff1ec1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1ec10a9b0 0x7ff1ec19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.282+0000 7ff1f4800640 1 --2- 172.21.15.103:0/2522873772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1ec104fa0 0x7ff1ec19b450 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.283+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 >> 172.21.15.103:0/2522873772 conn(0x7ff1ec100a50 msgr2=0x7ff1ec102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.283+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 shutdown_connections 2024-09-11T00:22:33.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:33.283+0000 7ff1f4800640 1 -- 172.21.15.103:0/2522873772 wait complete. 2024-09-11T00:22:33.495 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906983 got 274877906987 for osd.10 2024-09-11T00:22:33.495 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:34.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:33 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2522873772' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:22:34.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:33 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2522873772' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:22:34.241 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070511 got 253403070514 for osd.9 2024-09-11T00:22:34.241 DEBUG:teuthology.parallel:result is None 2024-09-11T00:22:34.241 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:22:34.241 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:22:34.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:33 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2522873772' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-11T00:22:34.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:34 smithi103 ceph-mon[30578]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T00:22:35.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:34 smithi117 ceph-mon[35525]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T00:22:35.206 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:35.280 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:34 smithi183 ceph-mon[35934]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-11T00:22:35.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 -- 172.21.15.103:0/3853063733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 msgr2=0x7fadf4115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:35.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 --2- 172.21.15.103:0/3853063733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 0x7fadf4115f10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fadec009ca0 tx=0x7fadec02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:22:35.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 -- 172.21.15.103:0/3853063733 shutdown_connections 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 --2- 172.21.15.103:0/3853063733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 0x7fadf4115f10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 --2- 172.21.15.103:0/3853063733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 0x7fadf41135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 --2- 172.21.15.103:0/3853063733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4075540 0x7fadf4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 -- 172.21.15.103:0/3853063733 >> 172.21.15.103:0/3853063733 conn(0x7fadf40fe830 msgr2=0x7fadf4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 -- 172.21.15.103:0/3853063733 shutdown_connections 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.843+0000 7fadfae00640 1 -- 172.21.15.103:0/3853063733 wait complete. 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.844+0000 7fadfae00640 1 Processor -- start 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.844+0000 7fadfae00640 1 -- start start 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadfae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4075540 0x7fadf419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadfae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 0x7fadf419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:35.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadfae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 0x7fadf41a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadfae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fadf41193e0 con 0x7fadf4113b20 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadfae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4119260 con 0x7fadf4075540 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadfae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadf4119560 con 0x7fadf4075e60 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadf3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 0x7fadf419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadf8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 0x7fadf41a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadf3400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 0x7fadf419fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:47554/0 (socket says 172.21.15.103:47554) 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadf3400640 1 -- 172.21.15.103:0/214327334 learned_addr learned my addr 172.21.15.103:0/214327334 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadf3e00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4075540 0x7fadf419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadf3400640 1 -- 172.21.15.103:0/214327334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4075540 msgr2=0x7fadf419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.845+0000 7fadf3400640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4075540 0x7fadf419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadf3400640 1 -- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 msgr2=0x7fadf41a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadf3400640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 0x7fadf41a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:35.854 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadf3400640 1 -- 172.21.15.103:0/214327334 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf41a5960 con 0x7fadf4075e60 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadf3400640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 0x7fadf419fdf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fade000e9f0 tx=0x7fade000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadf0c00640 1 -- 172.21.15.103:0/214327334 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade000cd80 con 0x7fadf4075e60 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadf0c00640 1 -- 172.21.15.103:0/214327334 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade00054d0 con 0x7fadf4075e60 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf41a5c50 con 0x7fadf4075e60 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadf0c00640 1 -- 172.21.15.103:0/214327334 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0010690 con 0x7fadf4075e60 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.846+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf41ac310 con 0x7fadf4075e60 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.847+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf4075920 con 0x7fadf4075e60 2024-09-11T00:22:35.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.850+0000 7fadf0c00640 1 -- 172.21.15.103:0/214327334 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fade0004e30 con 0x7fadf4075e60 2024-09-11T00:22:35.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.852+0000 7fadf0c00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fadd80b5370 0x7fadd80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:35.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.852+0000 7fadf0c00640 1 -- 172.21.15.103:0/214327334 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fade00e6730 con 0x7fadf4075e60 2024-09-11T00:22:35.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.852+0000 7fadf3e00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fadd80b5370 0x7fadd80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:35.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.852+0000 7fadf0c00640 1 -- 172.21.15.103:0/214327334 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fade00ad0a0 con 0x7fadf4075e60 2024-09-11T00:22:35.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:35.853+0000 7fadf3e00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fadd80b5370 0x7fadd80b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fade4009bd0 tx=0x7fade4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:36.048 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.045+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fadf4100b50 con 0x7fadd80b5370 2024-09-11T00:22:36.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.052+0000 7fadf0c00640 1 -- 172.21.15.103:0/214327334 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340357 (secure 0 0 0) 0x7fadf4100b50 con 0x7fadd80b5370 2024-09-11T00:22:36.058 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:22:36.060 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-11T00:22:36.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fadd80b5370 msgr2=0x7fadd80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:36.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fadd80b5370 0x7fadd80b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fade4009bd0 tx=0x7fade4005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:36.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 msgr2=0x7fadf419fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:36.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 0x7fadf419fdf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fade000e9f0 tx=0x7fade000edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 shutdown_connections 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fadd80b5370 0x7fadd80b7830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf4113b20 0x7fadf41a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadf4075e60 0x7fadf419fdf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 --2- 172.21.15.103:0/214327334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4075540 0x7fadf419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 >> 172.21.15.103:0/214327334 conn(0x7fadf40fe830 msgr2=0x7fadf40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 shutdown_connections 2024-09-11T00:22:36.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:36.057+0000 7fadfae00640 1 -- 172.21.15.103:0/214327334 wait complete. 2024-09-11T00:22:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:36 smithi183 ceph-mon[35934]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:22:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:36 smithi183 ceph-mon[35934]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-11T00:22:36.799 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:36 smithi103 ceph-mon[30578]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:22:36.799 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:36 smithi103 ceph-mon[30578]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-11T00:22:36.840 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-11T00:22:34.391488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656675,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":514,"num_read_kb":351,"num_write":294,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375884,"kb_used_data":14068,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565940,"statfs":{"total":1151940427776,"available":1151555522560,"internally_reserved":0,"allocated":14405632,"data_stored":8116661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":25,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002056"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680924+0000","last_change":"2024-09-11T00:20:24.027578+0000","last_active":"2024-09-11T00:22:28.680924+0000","last_peered":"2024-09-11T00:22:28.680924+0000","last_clean":"2024-09-11T00:22:28.680924+0000","last_became_active":"2024-09-11T00:20:24.027174+0000","last_became_peered":"2024-09-11T00:20:24.027174+0000","last_unstale":"2024-09-11T00:22:28.680924+0000","last_undegraded":"2024-09-11T00:22:28.680924+0000","last_fullsized":"2024-09-11T00:22:28.680924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:15.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863777+0000","last_change":"2024-09-11T00:20:27.064047+0000","last_active":"2024-09-11T00:22:28.863777+0000","last_peered":"2024-09-11T00:22:28.863777+0000","last_clean":"2024-09-11T00:22:28.863777+0000","last_became_active":"2024-09-11T00:20:27.063756+0000","last_became_peered":"2024-09-11T00:20:27.063756+0000","last_unstale":"2024-09-11T00:22:28.863777+0000","last_undegraded":"2024-09-11T00:22:28.863777+0000","last_fullsized":"2024-09-11T00:22:28.863777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.695888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679187+0000","last_change":"2024-09-11T00:20:37.449428+0000","last_active":"2024-09-11T00:22:28.679187+0000","last_peered":"2024-09-11T00:22:28.679187+0000","last_clean":"2024-09-11T00:22:28.679187+0000","last_became_active":"2024-09-11T00:20:37.449015+0000","last_became_peered":"2024-09-11T00:20:37.449015+0000","last_unstale":"2024-09-11T00:22:28.679187+0000","last_undegraded":"2024-09-11T00:22:28.679187+0000","last_fullsized":"2024-09-11T00:22:28.679187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:42.983990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864076+0000","last_change":"2024-09-11T00:20:24.035966+0000","last_active":"2024-09-11T00:22:28.864076+0000","last_peered":"2024-09-11T00:22:28.864076+0000","last_clean":"2024-09-11T00:22:28.864076+0000","last_became_active":"2024-09-11T00:20:24.035663+0000","last_became_peered":"2024-09-11T00:20:24.035663+0000","last_unstale":"2024-09-11T00:22:28.864076+0000","last_undegraded":"2024-09-11T00:22:28.864076+0000","last_fullsized":"2024-09-11T00:22:28.864076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:22.529392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679642+0000","last_change":"2024-09-11T00:20:27.066470+0000","last_active":"2024-09-11T00:22:28.679642+0000","last_peered":"2024-09-11T00:22:28.679642+0000","last_clean":"2024-09-11T00:22:28.679642+0000","last_became_active":"2024-09-11T00:20:27.065424+0000","last_became_peered":"2024-09-11T00:20:27.065424+0000","last_unstale":"2024-09-11T00:22:28.679642+0000","last_undegraded":"2024-09-11T00:22:28.679642+0000","last_fullsized":"2024-09-11T00:22:28.679642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:39.967332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864756+0000","last_change":"2024-09-11T00:20:37.453105+0000","last_active":"2024-09-11T00:22:28.864756+0000","last_peered":"2024-09-11T00:22:28.864756+0000","last_clean":"2024-09-11T00:22:28.864756+0000","last_became_active":"2024-09-11T00:20:37.452319+0000","last_became_peered":"2024-09-11T00:20:37.452319+0000","last_unstale":"2024-09-11T00:22:28.864756+0000","last_undegraded":"2024-09-11T00:22:28.864756+0000","last_fullsized":"2024-09-11T00:22:28.864756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:05.873046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864760+0000","last_change":"2024-09-11T00:20:24.022277+0000","last_active":"2024-09-11T00:22:28.864760+0000","last_peered":"2024-09-11T00:22:28.864760+0000","last_clean":"2024-09-11T00:22:28.864760+0000","last_became_active":"2024-09-11T00:20:24.021893+0000","last_became_peered":"2024-09-11T00:20:24.021893+0000","last_unstale":"2024-09-11T00:22:28.864760+0000","last_undegraded":"2024-09-11T00:22:28.864760+0000","last_fullsized":"2024-09-11T00:22:28.864760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.994032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864847+0000","last_change":"2024-09-11T00:20:27.063368+0000","last_active":"2024-09-11T00:22:28.864847+0000","last_peered":"2024-09-11T00:22:28.864847+0000","last_clean":"2024-09-11T00:22:28.864847+0000","last_became_active":"2024-09-11T00:20:27.063051+0000","last_became_peered":"2024-09-11T00:20:27.063051+0000","last_unstale":"2024-09-11T00:22:28.864847+0000","last_undegraded":"2024-09-11T00:22:28.864847+0000","last_fullsized":"2024-09-11T00:22:28.864847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:09.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864787+0000","last_change":"2024-09-11T00:21:28.229073+0000","last_active":"2024-09-11T00:22:28.864787+0000","last_peered":"2024-09-11T00:22:28.864787+0000","last_clean":"2024-09-11T00:22:28.864787+0000","last_became_active":"2024-09-11T00:21:28.228775+0000","last_became_peered":"2024-09-11T00:21:28.228775+0000","last_unstale":"2024-09-11T00:22:28.864787+0000","last_undegraded":"2024-09-11T00:22:28.864787+0000","last_fullsized":"2024-09-11T00:22:28.864787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.886727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864712+0000","last_change":"2024-09-11T00:20:24.022562+0000","last_active":"2024-09-11T00:22:28.864712+0000","last_peered":"2024-09-11T00:22:28.864712+0000","last_clean":"2024-09-11T00:22:28.864712+0000","last_became_active":"2024-09-11T00:20:24.022207+0000","last_became_peered":"2024-09-11T00:20:24.022207+0000","last_unstale":"2024-09-11T00:22:28.864712+0000","last_undegraded":"2024-09-11T00:22:28.864712+0000","last_fullsized":"2024-09-11T00:22:28.864712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:01.071034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734397+0000","last_change":"2024-09-11T00:20:27.062572+0000","last_active":"2024-09-11T00:22:28.734397+0000","last_peered":"2024-09-11T00:22:28.734397+0000","last_clean":"2024-09-11T00:22:28.734397+0000","last_became_active":"2024-09-11T00:20:27.062288+0000","last_became_peered":"2024-09-11T00:20:27.062288+0000","last_unstale":"2024-09-11T00:22:28.734397+0000","last_undegraded":"2024-09-11T00:22:28.734397+0000","last_fullsized":"2024-09-11T00:22:28.734397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:37.391822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864621+0000","last_change":"2024-09-11T00:20:37.444028+0000","last_active":"2024-09-11T00:22:28.864621+0000","last_peered":"2024-09-11T00:22:28.864621+0000","last_clean":"2024-09-11T00:22:28.864621+0000","last_became_active":"2024-09-11T00:20:37.443344+0000","last_became_peered":"2024-09-11T00:20:37.443344+0000","last_unstale":"2024-09-11T00:22:28.864621+0000","last_undegraded":"2024-09-11T00:22:28.864621+0000","last_fullsized":"2024-09-11T00:22:28.864621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.347054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866013+0000","last_change":"2024-09-11T00:20:27.060317+0000","last_active":"2024-09-11T00:22:28.866013+0000","last_peered":"2024-09-11T00:22:28.866013+0000","last_clean":"2024-09-11T00:22:28.866013+0000","last_became_active":"2024-09-11T00:20:27.059983+0000","last_became_peered":"2024-09-11T00:20:27.059983+0000","last_unstale":"2024-09-11T00:22:28.866013+0000","last_undegraded":"2024-09-11T00:22:28.866013+0000","last_fullsized":"2024-09-11T00:22:28.866013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:28.405553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864185+0000","last_change":"2024-09-11T00:20:24.025555+0000","last_active":"2024-09-11T00:22:28.864185+0000","last_peered":"2024-09-11T00:22:28.864185+0000","last_clean":"2024-09-11T00:22:28.864185+0000","last_became_active":"2024-09-11T00:20:24.025171+0000","last_became_peered":"2024-09-11T00:20:24.025171+0000","last_unstale":"2024-09-11T00:22:28.864185+0000","last_undegraded":"2024-09-11T00:22:28.864185+0000","last_fullsized":"2024-09-11T00:22:28.864185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:22.708985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864433+0000","last_change":"2024-09-11T00:20:37.442057+0000","last_active":"2024-09-11T00:22:28.864433+0000","last_peered":"2024-09-11T00:22:28.864433+0000","last_clean":"2024-09-11T00:22:28.864433+0000","last_became_active":"2024-09-11T00:20:37.441555+0000","last_became_peered":"2024-09-11T00:20:37.441555+0000","last_unstale":"2024-09-11T00:22:28.864433+0000","last_undegraded":"2024-09-11T00:22:28.864433+0000","last_fullsized":"2024-09-11T00:22:28.864433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:08.155763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863627+0000","last_change":"2024-09-11T00:20:27.071533+0000","last_active":"2024-09-11T00:22:28.863627+0000","last_peered":"2024-09-11T00:22:28.863627+0000","last_clean":"2024-09-11T00:22:28.863627+0000","last_became_active":"2024-09-11T00:20:27.068771+0000","last_became_peered":"2024-09-11T00:20:27.068771+0000","last_unstale":"2024-09-11T00:22:28.863627+0000","last_undegraded":"2024-09-11T00:22:28.863627+0000","last_fullsized":"2024-09-11T00:22:28.863627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:05.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679472+0000","last_change":"2024-09-11T00:20:24.023282+0000","last_active":"2024-09-11T00:22:28.679472+0000","last_peered":"2024-09-11T00:22:28.679472+0000","last_clean":"2024-09-11T00:22:28.679472+0000","last_became_active":"2024-09-11T00:20:24.022914+0000","last_became_peered":"2024-09-11T00:20:24.022914+0000","last_unstale":"2024-09-11T00:22:28.679472+0000","last_undegraded":"2024-09-11T00:22:28.679472+0000","last_fullsized":"2024-09-11T00:22:28.679472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:45.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864723+0000","last_change":"2024-09-11T00:20:37.457338+0000","last_active":"2024-09-11T00:22:28.864723+0000","last_peered":"2024-09-11T00:22:28.864723+0000","last_clean":"2024-09-11T00:22:28.864723+0000","last_became_active":"2024-09-11T00:20:37.456993+0000","last_became_peered":"2024-09-11T00:20:37.456993+0000","last_unstale":"2024-09-11T00:22:28.864723+0000","last_undegraded":"2024-09-11T00:22:28.864723+0000","last_fullsized":"2024-09-11T00:22:28.864723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:57.284412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864064+0000","last_change":"2024-09-11T00:20:27.062752+0000","last_active":"2024-09-11T00:22:28.864064+0000","last_peered":"2024-09-11T00:22:28.864064+0000","last_clean":"2024-09-11T00:22:28.864064+0000","last_became_active":"2024-09-11T00:20:27.061569+0000","last_became_peered":"2024-09-11T00:20:27.061569+0000","last_unstale":"2024-09-11T00:22:28.864064+0000","last_undegraded":"2024-09-11T00:22:28.864064+0000","last_fullsized":"2024-09-11T00:22:28.864064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:30.904423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864019+0000","last_change":"2024-09-11T00:20:24.023197+0000","last_active":"2024-09-11T00:22:28.864019+0000","last_peered":"2024-09-11T00:22:28.864019+0000","last_clean":"2024-09-11T00:22:28.864019+0000","last_became_active":"2024-09-11T00:20:24.022867+0000","last_became_peered":"2024-09-11T00:20:24.022867+0000","last_unstale":"2024-09-11T00:22:28.864019+0000","last_undegraded":"2024-09-11T00:22:28.864019+0000","last_fullsized":"2024-09-11T00:22:28.864019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:37.949437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680610+0000","last_change":"2024-09-11T00:20:37.451731+0000","last_active":"2024-09-11T00:22:28.680610+0000","last_peered":"2024-09-11T00:22:28.680610+0000","last_clean":"2024-09-11T00:22:28.680610+0000","last_became_active":"2024-09-11T00:20:37.447710+0000","last_became_peered":"2024-09-11T00:20:37.447710+0000","last_unstale":"2024-09-11T00:22:28.680610+0000","last_undegraded":"2024-09-11T00:22:28.680610+0000","last_fullsized":"2024-09-11T00:22:28.680610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:11:38.471216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863635+0000","last_change":"2024-09-11T00:20:27.066021+0000","last_active":"2024-09-11T00:22:28.863635+0000","last_peered":"2024-09-11T00:22:28.863635+0000","last_clean":"2024-09-11T00:22:28.863635+0000","last_became_active":"2024-09-11T00:20:27.065652+0000","last_became_peered":"2024-09-11T00:20:27.065652+0000","last_unstale":"2024-09-11T00:22:28.863635+0000","last_undegraded":"2024-09-11T00:22:28.863635+0000","last_fullsized":"2024-09-11T00:22:28.863635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:01.933031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864083+0000","last_change":"2024-09-11T00:20:24.035619+0000","last_active":"2024-09-11T00:22:28.864083+0000","last_peered":"2024-09-11T00:22:28.864083+0000","last_clean":"2024-09-11T00:22:28.864083+0000","last_became_active":"2024-09-11T00:20:24.035358+0000","last_became_peered":"2024-09-11T00:20:24.035358+0000","last_unstale":"2024-09-11T00:22:28.864083+0000","last_undegraded":"2024-09-11T00:22:28.864083+0000","last_fullsized":"2024-09-11T00:22:28.864083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:56.890058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864427+0000","last_change":"2024-09-11T00:21:28.226415+0000","last_active":"2024-09-11T00:22:28.864427+0000","last_peered":"2024-09-11T00:22:28.864427+0000","last_clean":"2024-09-11T00:22:28.864427+0000","last_became_active":"2024-09-11T00:21:28.225894+0000","last_became_peered":"2024-09-11T00:21:28.225894+0000","last_unstale":"2024-09-11T00:22:28.864427+0000","last_undegraded":"2024-09-11T00:22:28.864427+0000","last_fullsized":"2024-09-11T00:22:28.864427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:59.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866953+0000","last_change":"2024-09-11T00:22:28.866891+0000","last_active":"2024-09-11T00:22:28.866953+0000","last_peered":"2024-09-11T00:22:28.866953+0000","last_clean":"2024-09-11T00:22:28.866953+0000","last_became_active":"2024-09-11T00:22:28.866541+0000","last_became_peered":"2024-09-11T00:22:28.866541+0000","last_unstale":"2024-09-11T00:22:28.866953+0000","last_undegraded":"2024-09-11T00:22:28.866953+0000","last_fullsized":"2024-09-11T00:22:28.866953+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:35.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865567+0000","last_change":"2024-09-11T00:20:24.535701+0000","last_active":"2024-09-11T00:22:28.865567+0000","last_peered":"2024-09-11T00:22:28.865567+0000","last_clean":"2024-09-11T00:22:28.865567+0000","last_became_active":"2024-09-11T00:20:24.535389+0000","last_became_peered":"2024-09-11T00:20:24.535389+0000","last_unstale":"2024-09-11T00:22:28.865567+0000","last_undegraded":"2024-09-11T00:22:28.865567+0000","last_fullsized":"2024-09-11T00:22:28.865567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:29.044747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679581+0000","last_change":"2024-09-11T00:20:37.435584+0000","last_active":"2024-09-11T00:22:28.679581+0000","last_peered":"2024-09-11T00:22:28.679581+0000","last_clean":"2024-09-11T00:22:28.679581+0000","last_became_active":"2024-09-11T00:20:37.435237+0000","last_became_peered":"2024-09-11T00:20:37.435237+0000","last_unstale":"2024-09-11T00:22:28.679581+0000","last_undegraded":"2024-09-11T00:22:28.679581+0000","last_fullsized":"2024-09-11T00:22:28.679581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:34.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863962+0000","last_change":"2024-09-11T00:20:27.068718+0000","last_active":"2024-09-11T00:22:28.863962+0000","last_peered":"2024-09-11T00:22:28.863962+0000","last_clean":"2024-09-11T00:22:28.863962+0000","last_became_active":"2024-09-11T00:20:27.058921+0000","last_became_peered":"2024-09-11T00:20:27.058921+0000","last_unstale":"2024-09-11T00:22:28.863962+0000","last_undegraded":"2024-09-11T00:22:28.863962+0000","last_fullsized":"2024-09-11T00:22:28.863962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:16.056742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680891+0000","last_change":"2024-09-11T00:20:24.023580+0000","last_active":"2024-09-11T00:22:28.680891+0000","last_peered":"2024-09-11T00:22:28.680891+0000","last_clean":"2024-09-11T00:22:28.680891+0000","last_became_active":"2024-09-11T00:20:24.023029+0000","last_became_peered":"2024-09-11T00:20:24.023029+0000","last_unstale":"2024-09-11T00:22:28.680891+0000","last_undegraded":"2024-09-11T00:22:28.680891+0000","last_fullsized":"2024-09-11T00:22:28.680891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:05.574228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863409+0000","last_change":"2024-09-11T00:20:37.449229+0000","last_active":"2024-09-11T00:22:28.863409+0000","last_peered":"2024-09-11T00:22:28.863409+0000","last_clean":"2024-09-11T00:22:28.863409+0000","last_became_active":"2024-09-11T00:20:37.447464+0000","last_became_peered":"2024-09-11T00:20:37.447464+0000","last_unstale":"2024-09-11T00:22:28.863409+0000","last_undegraded":"2024-09-11T00:22:28.863409+0000","last_fullsized":"2024-09-11T00:22:28.863409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:47.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683028+0000","last_change":"2024-09-11T00:20:27.063518+0000","last_active":"2024-09-11T00:22:28.683028+0000","last_peered":"2024-09-11T00:22:28.683028+0000","last_clean":"2024-09-11T00:22:28.683028+0000","last_became_active":"2024-09-11T00:20:27.063122+0000","last_became_peered":"2024-09-11T00:20:27.063122+0000","last_unstale":"2024-09-11T00:22:28.683028+0000","last_undegraded":"2024-09-11T00:22:28.683028+0000","last_fullsized":"2024-09-11T00:22:28.683028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:52.269862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864785+0000","last_change":"2024-09-11T00:20:24.035835+0000","last_active":"2024-09-11T00:22:28.864785+0000","last_peered":"2024-09-11T00:22:28.864785+0000","last_clean":"2024-09-11T00:22:28.864785+0000","last_became_active":"2024-09-11T00:20:24.035471+0000","last_became_peered":"2024-09-11T00:20:24.035471+0000","last_unstale":"2024-09-11T00:22:28.864785+0000","last_undegraded":"2024-09-11T00:22:28.864785+0000","last_fullsized":"2024-09-11T00:22:28.864785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:37.114863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863849+0000","last_change":"2024-09-11T00:20:37.439609+0000","last_active":"2024-09-11T00:22:28.863849+0000","last_peered":"2024-09-11T00:22:28.863849+0000","last_clean":"2024-09-11T00:22:28.863849+0000","last_became_active":"2024-09-11T00:20:37.439139+0000","last_became_peered":"2024-09-11T00:20:37.439139+0000","last_unstale":"2024-09-11T00:22:28.863849+0000","last_undegraded":"2024-09-11T00:22:28.863849+0000","last_fullsized":"2024-09-11T00:22:28.863849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:19.152096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830271+0000","last_change":"2024-09-11T00:20:27.062879+0000","last_active":"2024-09-11T00:22:28.830271+0000","last_peered":"2024-09-11T00:22:28.830271+0000","last_clean":"2024-09-11T00:22:28.830271+0000","last_became_active":"2024-09-11T00:20:27.062200+0000","last_became_peered":"2024-09-11T00:20:27.062200+0000","last_unstale":"2024-09-11T00:22:28.830271+0000","last_undegraded":"2024-09-11T00:22:28.830271+0000","last_fullsized":"2024-09-11T00:22:28.830271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:57.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680751+0000","last_change":"2024-09-11T00:21:28.230802+0000","last_active":"2024-09-11T00:22:28.680751+0000","last_peered":"2024-09-11T00:22:28.680751+0000","last_clean":"2024-09-11T00:22:28.680751+0000","last_became_active":"2024-09-11T00:21:28.230378+0000","last_became_peered":"2024-09-11T00:21:28.230378+0000","last_unstale":"2024-09-11T00:22:28.680751+0000","last_undegraded":"2024-09-11T00:22:28.680751+0000","last_fullsized":"2024-09-11T00:22:28.680751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:35.676964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864366+0000","last_change":"2024-09-11T00:20:37.444912+0000","last_active":"2024-09-11T00:22:28.864366+0000","last_peered":"2024-09-11T00:22:28.864366+0000","last_clean":"2024-09-11T00:22:28.864366+0000","last_became_active":"2024-09-11T00:20:37.444472+0000","last_became_peered":"2024-09-11T00:20:37.444472+0000","last_unstale":"2024-09-11T00:22:28.864366+0000","last_undegraded":"2024-09-11T00:22:28.864366+0000","last_fullsized":"2024-09-11T00:22:28.864366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:23.263981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830293+0000","last_change":"2024-09-11T00:20:27.060477+0000","last_active":"2024-09-11T00:22:28.830293+0000","last_peered":"2024-09-11T00:22:28.830293+0000","last_clean":"2024-09-11T00:22:28.830293+0000","last_became_active":"2024-09-11T00:20:27.060128+0000","last_became_peered":"2024-09-11T00:20:27.060128+0000","last_unstale":"2024-09-11T00:22:28.830293+0000","last_undegraded":"2024-09-11T00:22:28.830293+0000","last_fullsized":"2024-09-11T00:22:28.830293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:55.967799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866156+0000","last_change":"2024-09-11T00:20:24.028229+0000","last_active":"2024-09-11T00:22:28.866156+0000","last_peered":"2024-09-11T00:22:28.866156+0000","last_clean":"2024-09-11T00:22:28.866156+0000","last_became_active":"2024-09-11T00:20:24.027832+0000","last_became_peered":"2024-09-11T00:20:24.027832+0000","last_unstale":"2024-09-11T00:22:28.866156+0000","last_undegraded":"2024-09-11T00:22:28.866156+0000","last_fullsized":"2024-09-11T00:22:28.866156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:19.960403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682825+0000","last_change":"2024-09-11T00:20:37.442414+0000","last_active":"2024-09-11T00:22:28.682825+0000","last_peered":"2024-09-11T00:22:28.682825+0000","last_clean":"2024-09-11T00:22:28.682825+0000","last_became_active":"2024-09-11T00:20:37.442021+0000","last_became_peered":"2024-09-11T00:20:37.442021+0000","last_unstale":"2024-09-11T00:22:28.682825+0000","last_undegraded":"2024-09-11T00:22:28.682825+0000","last_fullsized":"2024-09-11T00:22:28.682825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:21.296638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680259+0000","last_change":"2024-09-11T00:21:28.233303+0000","last_active":"2024-09-11T00:22:28.680259+0000","last_peered":"2024-09-11T00:22:28.680259+0000","last_clean":"2024-09-11T00:22:28.680259+0000","last_became_active":"2024-09-11T00:21:28.232039+0000","last_became_peered":"2024-09-11T00:21:28.232039+0000","last_unstale":"2024-09-11T00:22:28.680259+0000","last_undegraded":"2024-09-11T00:22:28.680259+0000","last_fullsized":"2024-09-11T00:22:28.680259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:36.828102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":115,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:30.315803+0000","last_change":"2024-09-11T00:20:24.037218+0000","last_active":"2024-09-11T00:22:30.315803+0000","last_peered":"2024-09-11T00:22:30.315803+0000","last_clean":"2024-09-11T00:22:30.315803+0000","last_became_active":"2024-09-11T00:20:24.036005+0000","last_became_peered":"2024-09-11T00:20:24.036005+0000","last_unstale":"2024-09-11T00:22:30.315803+0000","last_undegraded":"2024-09-11T00:22:30.315803+0000","last_fullsized":"2024-09-11T00:22:30.315803+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:17.758812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683029+0000","last_change":"2024-09-11T00:20:37.445584+0000","last_active":"2024-09-11T00:22:28.683029+0000","last_peered":"2024-09-11T00:22:28.683029+0000","last_clean":"2024-09-11T00:22:28.683029+0000","last_became_active":"2024-09-11T00:20:37.445196+0000","last_became_peered":"2024-09-11T00:20:37.445196+0000","last_unstale":"2024-09-11T00:22:28.683029+0000","last_undegraded":"2024-09-11T00:22:28.683029+0000","last_fullsized":"2024-09-11T00:22:28.683029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:05.966282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682578+0000","last_change":"2024-09-11T00:20:27.060428+0000","last_active":"2024-09-11T00:22:28.682578+0000","last_peered":"2024-09-11T00:22:28.682578+0000","last_clean":"2024-09-11T00:22:28.682578+0000","last_became_active":"2024-09-11T00:20:27.060057+0000","last_became_peered":"2024-09-11T00:20:27.060057+0000","last_unstale":"2024-09-11T00:22:28.682578+0000","last_undegraded":"2024-09-11T00:22:28.682578+0000","last_fullsized":"2024-09-11T00:22:28.682578+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:37.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865091+0000","last_change":"2024-09-11T00:20:24.025496+0000","last_active":"2024-09-11T00:22:28.865091+0000","last_peered":"2024-09-11T00:22:28.865091+0000","last_clean":"2024-09-11T00:22:28.865091+0000","last_became_active":"2024-09-11T00:20:24.025158+0000","last_became_peered":"2024-09-11T00:20:24.025158+0000","last_unstale":"2024-09-11T00:22:28.865091+0000","last_undegraded":"2024-09-11T00:22:28.865091+0000","last_fullsized":"2024-09-11T00:22:28.865091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:40.439708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734388+0000","last_change":"2024-09-11T00:20:37.453118+0000","last_active":"2024-09-11T00:22:28.734388+0000","last_peered":"2024-09-11T00:22:28.734388+0000","last_clean":"2024-09-11T00:22:28.734388+0000","last_became_active":"2024-09-11T00:20:37.452669+0000","last_became_peered":"2024-09-11T00:20:37.452669+0000","last_unstale":"2024-09-11T00:22:28.734388+0000","last_undegraded":"2024-09-11T00:22:28.734388+0000","last_fullsized":"2024-09-11T00:22:28.734388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:51:58.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863822+0000","last_change":"2024-09-11T00:20:27.069708+0000","last_active":"2024-09-11T00:22:28.863822+0000","last_peered":"2024-09-11T00:22:28.863822+0000","last_clean":"2024-09-11T00:22:28.863822+0000","last_became_active":"2024-09-11T00:20:27.067500+0000","last_became_peered":"2024-09-11T00:20:27.067500+0000","last_unstale":"2024-09-11T00:22:28.863822+0000","last_undegraded":"2024-09-11T00:22:28.863822+0000","last_fullsized":"2024-09-11T00:22:28.863822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:57.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864981+0000","last_change":"2024-09-11T00:20:24.023504+0000","last_active":"2024-09-11T00:22:28.864981+0000","last_peered":"2024-09-11T00:22:28.864981+0000","last_clean":"2024-09-11T00:22:28.864981+0000","last_became_active":"2024-09-11T00:20:24.022706+0000","last_became_peered":"2024-09-11T00:20:24.022706+0000","last_unstale":"2024-09-11T00:22:28.864981+0000","last_undegraded":"2024-09-11T00:22:28.864981+0000","last_fullsized":"2024-09-11T00:22:28.864981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:21.216325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865351+0000","last_change":"2024-09-11T00:19:05.531191+0000","last_active":"2024-09-11T00:22:28.865351+0000","last_peered":"2024-09-11T00:22:28.865351+0000","last_clean":"2024-09-11T00:22:28.865351+0000","last_became_active":"2024-09-11T00:19:05.525664+0000","last_became_peered":"2024-09-11T00:19:05.525664+0000","last_unstale":"2024-09-11T00:22:28.865351+0000","last_undegraded":"2024-09-11T00:22:28.865351+0000","last_fullsized":"2024-09-11T00:22:28.865351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_clean_scrub_stamp":"2024-09-11T00:15:23.365345+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:06.253537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865876+0000","last_change":"2024-09-11T00:20:37.446403+0000","last_active":"2024-09-11T00:22:28.865876+0000","last_peered":"2024-09-11T00:22:28.865876+0000","last_clean":"2024-09-11T00:22:28.865876+0000","last_became_active":"2024-09-11T00:20:37.446029+0000","last_became_peered":"2024-09-11T00:20:37.446029+0000","last_unstale":"2024-09-11T00:22:28.865876+0000","last_undegraded":"2024-09-11T00:22:28.865876+0000","last_fullsized":"2024-09-11T00:22:28.865876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866106+0000","last_change":"2024-09-11T00:20:27.063086+0000","last_active":"2024-09-11T00:22:28.866106+0000","last_peered":"2024-09-11T00:22:28.866106+0000","last_clean":"2024-09-11T00:22:28.866106+0000","last_became_active":"2024-09-11T00:20:27.062790+0000","last_became_peered":"2024-09-11T00:20:27.062790+0000","last_unstale":"2024-09-11T00:22:28.866106+0000","last_undegraded":"2024-09-11T00:22:28.866106+0000","last_fullsized":"2024-09-11T00:22:28.866106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:36.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679590+0000","last_change":"2024-09-11T00:20:24.535837+0000","last_active":"2024-09-11T00:22:28.679590+0000","last_peered":"2024-09-11T00:22:28.679590+0000","last_clean":"2024-09-11T00:22:28.679590+0000","last_became_active":"2024-09-11T00:20:24.535308+0000","last_became_peered":"2024-09-11T00:20:24.535308+0000","last_unstale":"2024-09-11T00:22:28.679590+0000","last_undegraded":"2024-09-11T00:22:28.679590+0000","last_fullsized":"2024-09-11T00:22:28.679590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.807570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864182+0000","last_change":"2024-09-11T00:20:37.443844+0000","last_active":"2024-09-11T00:22:28.864182+0000","last_peered":"2024-09-11T00:22:28.864182+0000","last_clean":"2024-09-11T00:22:28.864182+0000","last_became_active":"2024-09-11T00:20:37.443374+0000","last_became_peered":"2024-09-11T00:20:37.443374+0000","last_unstale":"2024-09-11T00:22:28.864182+0000","last_undegraded":"2024-09-11T00:22:28.864182+0000","last_fullsized":"2024-09-11T00:22:28.864182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:47.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865704+0000","last_change":"2024-09-11T00:20:27.072874+0000","last_active":"2024-09-11T00:22:28.865704+0000","last_peered":"2024-09-11T00:22:28.865704+0000","last_clean":"2024-09-11T00:22:28.865704+0000","last_became_active":"2024-09-11T00:20:27.072454+0000","last_became_peered":"2024-09-11T00:20:27.072454+0000","last_unstale":"2024-09-11T00:22:28.865704+0000","last_undegraded":"2024-09-11T00:22:28.865704+0000","last_fullsized":"2024-09-11T00:22:28.865704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:25.731037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734627+0000","last_change":"2024-09-11T00:20:24.018929+0000","last_active":"2024-09-11T00:22:28.734627+0000","last_peered":"2024-09-11T00:22:28.734627+0000","last_clean":"2024-09-11T00:22:28.734627+0000","last_became_active":"2024-09-11T00:20:24.018575+0000","last_became_peered":"2024-09-11T00:20:24.018575+0000","last_unstale":"2024-09-11T00:22:28.734627+0000","last_undegraded":"2024-09-11T00:22:28.734627+0000","last_fullsized":"2024-09-11T00:22:28.734627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.695561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679036+0000","last_change":"2024-09-11T00:20:37.453468+0000","last_active":"2024-09-11T00:22:28.679036+0000","last_peered":"2024-09-11T00:22:28.679036+0000","last_clean":"2024-09-11T00:22:28.679036+0000","last_became_active":"2024-09-11T00:20:37.453007+0000","last_became_peered":"2024-09-11T00:20:37.453007+0000","last_unstale":"2024-09-11T00:22:28.679036+0000","last_undegraded":"2024-09-11T00:22:28.679036+0000","last_fullsized":"2024-09-11T00:22:28.679036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:22.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865897+0000","last_change":"2024-09-11T00:20:27.064618+0000","last_active":"2024-09-11T00:22:28.865897+0000","last_peered":"2024-09-11T00:22:28.865897+0000","last_clean":"2024-09-11T00:22:28.865897+0000","last_became_active":"2024-09-11T00:20:27.064310+0000","last_became_peered":"2024-09-11T00:20:27.064310+0000","last_unstale":"2024-09-11T00:22:28.865897+0000","last_undegraded":"2024-09-11T00:22:28.865897+0000","last_fullsized":"2024-09-11T00:22:28.865897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:41.320561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682985+0000","last_change":"2024-09-11T00:20:24.023522+0000","last_active":"2024-09-11T00:22:28.682985+0000","last_peered":"2024-09-11T00:22:28.682985+0000","last_clean":"2024-09-11T00:22:28.682985+0000","last_became_active":"2024-09-11T00:20:24.022694+0000","last_became_peered":"2024-09-11T00:20:24.022694+0000","last_unstale":"2024-09-11T00:22:28.682985+0000","last_undegraded":"2024-09-11T00:22:28.682985+0000","last_fullsized":"2024-09-11T00:22:28.682985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:12.374179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733868+0000","last_change":"2024-09-11T00:20:37.441239+0000","last_active":"2024-09-11T00:22:28.733868+0000","last_peered":"2024-09-11T00:22:28.733868+0000","last_clean":"2024-09-11T00:22:28.733868+0000","last_became_active":"2024-09-11T00:20:37.440825+0000","last_became_peered":"2024-09-11T00:20:37.440825+0000","last_unstale":"2024-09-11T00:22:28.733868+0000","last_undegraded":"2024-09-11T00:22:28.733868+0000","last_fullsized":"2024-09-11T00:22:28.733868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:04.972569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864299+0000","last_change":"2024-09-11T00:20:27.059780+0000","last_active":"2024-09-11T00:22:28.864299+0000","last_peered":"2024-09-11T00:22:28.864299+0000","last_clean":"2024-09-11T00:22:28.864299+0000","last_became_active":"2024-09-11T00:20:27.059411+0000","last_became_peered":"2024-09-11T00:20:27.059411+0000","last_unstale":"2024-09-11T00:22:28.864299+0000","last_undegraded":"2024-09-11T00:22:28.864299+0000","last_fullsized":"2024-09-11T00:22:28.864299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:42.058883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864457+0000","last_change":"2024-09-11T00:20:24.024115+0000","last_active":"2024-09-11T00:22:28.864457+0000","last_peered":"2024-09-11T00:22:28.864457+0000","last_clean":"2024-09-11T00:22:28.864457+0000","last_became_active":"2024-09-11T00:20:24.023729+0000","last_became_peered":"2024-09-11T00:20:24.023729+0000","last_unstale":"2024-09-11T00:22:28.864457+0000","last_undegraded":"2024-09-11T00:22:28.864457+0000","last_fullsized":"2024-09-11T00:22:28.864457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:15.874168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733847+0000","last_change":"2024-09-11T00:20:37.443591+0000","last_active":"2024-09-11T00:22:28.733847+0000","last_peered":"2024-09-11T00:22:28.733847+0000","last_clean":"2024-09-11T00:22:28.733847+0000","last_became_active":"2024-09-11T00:20:37.443115+0000","last_became_peered":"2024-09-11T00:20:37.443115+0000","last_unstale":"2024-09-11T00:22:28.733847+0000","last_undegraded":"2024-09-11T00:22:28.733847+0000","last_fullsized":"2024-09-11T00:22:28.733847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:34.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734026+0000","last_change":"2024-09-11T00:20:27.061676+0000","last_active":"2024-09-11T00:22:28.734026+0000","last_peered":"2024-09-11T00:22:28.734026+0000","last_clean":"2024-09-11T00:22:28.734026+0000","last_became_active":"2024-09-11T00:20:27.061372+0000","last_became_peered":"2024-09-11T00:20:27.061372+0000","last_unstale":"2024-09-11T00:22:28.734026+0000","last_undegraded":"2024-09-11T00:22:28.734026+0000","last_fullsized":"2024-09-11T00:22:28.734026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:24.296673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863776+0000","last_change":"2024-09-11T00:21:28.244227+0000","last_active":"2024-09-11T00:22:28.863776+0000","last_peered":"2024-09-11T00:22:28.863776+0000","last_clean":"2024-09-11T00:22:28.863776+0000","last_became_active":"2024-09-11T00:21:28.231547+0000","last_became_peered":"2024-09-11T00:21:28.231547+0000","last_unstale":"2024-09-11T00:22:28.863776+0000","last_undegraded":"2024-09-11T00:22:28.863776+0000","last_fullsized":"2024-09-11T00:22:28.863776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:35.119308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682940+0000","last_change":"2024-09-11T00:20:37.454603+0000","last_active":"2024-09-11T00:22:28.682940+0000","last_peered":"2024-09-11T00:22:28.682940+0000","last_clean":"2024-09-11T00:22:28.682940+0000","last_became_active":"2024-09-11T00:20:37.454123+0000","last_became_peered":"2024-09-11T00:20:37.454123+0000","last_unstale":"2024-09-11T00:22:28.682940+0000","last_undegraded":"2024-09-11T00:22:28.682940+0000","last_fullsized":"2024-09-11T00:22:28.682940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:09.111768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864882+0000","last_change":"2024-09-11T00:20:27.058806+0000","last_active":"2024-09-11T00:22:28.864882+0000","last_peered":"2024-09-11T00:22:28.864882+0000","last_clean":"2024-09-11T00:22:28.864882+0000","last_became_active":"2024-09-11T00:20:27.058458+0000","last_became_peered":"2024-09-11T00:20:27.058458+0000","last_unstale":"2024-09-11T00:22:28.864882+0000","last_undegraded":"2024-09-11T00:22:28.864882+0000","last_fullsized":"2024-09-11T00:22:28.864882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:48.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:32.318842+0000","last_change":"2024-09-11T00:20:24.022328+0000","last_active":"2024-09-11T00:22:32.318842+0000","last_peered":"2024-09-11T00:22:32.318842+0000","last_clean":"2024-09-11T00:22:32.318842+0000","last_became_active":"2024-09-11T00:20:24.021908+0000","last_became_peered":"2024-09-11T00:20:24.021908+0000","last_unstale":"2024-09-11T00:22:32.318842+0000","last_undegraded":"2024-09-11T00:22:32.318842+0000","last_fullsized":"2024-09-11T00:22:32.318842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:35.688767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680728+0000","last_change":"2024-09-11T00:20:37.444898+0000","last_active":"2024-09-11T00:22:28.680728+0000","last_peered":"2024-09-11T00:22:28.680728+0000","last_clean":"2024-09-11T00:22:28.680728+0000","last_became_active":"2024-09-11T00:20:37.443840+0000","last_became_peered":"2024-09-11T00:20:37.443840+0000","last_unstale":"2024-09-11T00:22:28.680728+0000","last_undegraded":"2024-09-11T00:22:28.680728+0000","last_fullsized":"2024-09-11T00:22:28.680728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:58.668882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865608+0000","last_change":"2024-09-11T00:20:27.061198+0000","last_active":"2024-09-11T00:22:28.865608+0000","last_peered":"2024-09-11T00:22:28.865608+0000","last_clean":"2024-09-11T00:22:28.865608+0000","last_became_active":"2024-09-11T00:20:27.060877+0000","last_became_peered":"2024-09-11T00:20:27.060877+0000","last_unstale":"2024-09-11T00:22:28.865608+0000","last_undegraded":"2024-09-11T00:22:28.865608+0000","last_fullsized":"2024-09-11T00:22:28.865608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:03.595262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734204+0000","last_change":"2024-09-11T00:20:24.022755+0000","last_active":"2024-09-11T00:22:28.734204+0000","last_peered":"2024-09-11T00:22:28.734204+0000","last_clean":"2024-09-11T00:22:28.734204+0000","last_became_active":"2024-09-11T00:20:24.021530+0000","last_became_peered":"2024-09-11T00:20:24.021530+0000","last_unstale":"2024-09-11T00:22:28.734204+0000","last_undegraded":"2024-09-11T00:22:28.734204+0000","last_fullsized":"2024-09-11T00:22:28.734204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:33.881000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864289+0000","last_change":"2024-09-11T00:20:37.443722+0000","last_active":"2024-09-11T00:22:28.864289+0000","last_peered":"2024-09-11T00:22:28.864289+0000","last_clean":"2024-09-11T00:22:28.864289+0000","last_became_active":"2024-09-11T00:20:37.443339+0000","last_became_peered":"2024-09-11T00:20:37.443339+0000","last_unstale":"2024-09-11T00:22:28.864289+0000","last_undegraded":"2024-09-11T00:22:28.864289+0000","last_fullsized":"2024-09-11T00:22:28.864289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:30.221804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682691+0000","last_change":"2024-09-11T00:20:27.073158+0000","last_active":"2024-09-11T00:22:28.682691+0000","last_peered":"2024-09-11T00:22:28.682691+0000","last_clean":"2024-09-11T00:22:28.682691+0000","last_became_active":"2024-09-11T00:20:27.072805+0000","last_became_peered":"2024-09-11T00:20:27.072805+0000","last_unstale":"2024-09-11T00:22:28.682691+0000","last_undegraded":"2024-09-11T00:22:28.682691+0000","last_fullsized":"2024-09-11T00:22:28.682691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:35.124632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679649+0000","last_change":"2024-09-11T00:20:24.027178+0000","last_active":"2024-09-11T00:22:28.679649+0000","last_peered":"2024-09-11T00:22:28.679649+0000","last_clean":"2024-09-11T00:22:28.679649+0000","last_became_active":"2024-09-11T00:20:24.026821+0000","last_became_peered":"2024-09-11T00:20:24.026821+0000","last_unstale":"2024-09-11T00:22:28.679649+0000","last_undegraded":"2024-09-11T00:22:28.679649+0000","last_fullsized":"2024-09-11T00:22:28.679649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:38.393135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734368+0000","last_change":"2024-09-11T00:20:37.447587+0000","last_active":"2024-09-11T00:22:28.734368+0000","last_peered":"2024-09-11T00:22:28.734368+0000","last_clean":"2024-09-11T00:22:28.734368+0000","last_became_active":"2024-09-11T00:20:37.447166+0000","last_became_peered":"2024-09-11T00:20:37.447166+0000","last_unstale":"2024-09-11T00:22:28.734368+0000","last_undegraded":"2024-09-11T00:22:28.734368+0000","last_fullsized":"2024-09-11T00:22:28.734368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:47.469539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864619+0000","last_change":"2024-09-11T00:20:27.059914+0000","last_active":"2024-09-11T00:22:28.864619+0000","last_peered":"2024-09-11T00:22:28.864619+0000","last_clean":"2024-09-11T00:22:28.864619+0000","last_became_active":"2024-09-11T00:20:27.059595+0000","last_became_peered":"2024-09-11T00:20:27.059595+0000","last_unstale":"2024-09-11T00:22:28.864619+0000","last_undegraded":"2024-09-11T00:22:28.864619+0000","last_fullsized":"2024-09-11T00:22:28.864619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.360010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:33.333900+0000","last_change":"2024-09-11T00:20:24.023094+0000","last_active":"2024-09-11T00:22:33.333900+0000","last_peered":"2024-09-11T00:22:33.333900+0000","last_clean":"2024-09-11T00:22:33.333900+0000","last_became_active":"2024-09-11T00:20:24.022742+0000","last_became_peered":"2024-09-11T00:20:24.022742+0000","last_unstale":"2024-09-11T00:22:33.333900+0000","last_undegraded":"2024-09-11T00:22:33.333900+0000","last_fullsized":"2024-09-11T00:22:33.333900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:41.354435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":147,"num_read_kb":134,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680544+0000","last_change":"2024-09-11T00:20:37.451610+0000","last_active":"2024-09-11T00:22:28.680544+0000","last_peered":"2024-09-11T00:22:28.680544+0000","last_clean":"2024-09-11T00:22:28.680544+0000","last_became_active":"2024-09-11T00:20:37.444233+0000","last_became_peered":"2024-09-11T00:20:37.444233+0000","last_unstale":"2024-09-11T00:22:28.680544+0000","last_undegraded":"2024-09-11T00:22:28.680544+0000","last_fullsized":"2024-09-11T00:22:28.680544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:02.558457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864954+0000","last_change":"2024-09-11T00:20:27.061229+0000","last_active":"2024-09-11T00:22:28.864954+0000","last_peered":"2024-09-11T00:22:28.864954+0000","last_clean":"2024-09-11T00:22:28.864954+0000","last_became_active":"2024-09-11T00:20:27.060360+0000","last_became_peered":"2024-09-11T00:20:27.060360+0000","last_unstale":"2024-09-11T00:22:28.864954+0000","last_undegraded":"2024-09-11T00:22:28.864954+0000","last_fullsized":"2024-09-11T00:22:28.864954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.107405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734515+0000","last_change":"2024-09-11T00:20:24.036483+0000","last_active":"2024-09-11T00:22:28.734515+0000","last_peered":"2024-09-11T00:22:28.734515+0000","last_clean":"2024-09-11T00:22:28.734515+0000","last_became_active":"2024-09-11T00:20:24.035855+0000","last_became_peered":"2024-09-11T00:20:24.035855+0000","last_unstale":"2024-09-11T00:22:28.734515+0000","last_undegraded":"2024-09-11T00:22:28.734515+0000","last_fullsized":"2024-09-11T00:22:28.734515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.611169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864426+0000","last_change":"2024-09-11T00:21:28.245665+0000","last_active":"2024-09-11T00:22:28.864426+0000","last_peered":"2024-09-11T00:22:28.864426+0000","last_clean":"2024-09-11T00:22:28.864426+0000","last_became_active":"2024-09-11T00:21:28.245221+0000","last_became_peered":"2024-09-11T00:21:28.245221+0000","last_unstale":"2024-09-11T00:22:28.864426+0000","last_undegraded":"2024-09-11T00:22:28.864426+0000","last_fullsized":"2024-09-11T00:22:28.864426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:36.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679687+0000","last_change":"2024-09-11T00:21:28.231435+0000","last_active":"2024-09-11T00:22:28.679687+0000","last_peered":"2024-09-11T00:22:28.679687+0000","last_clean":"2024-09-11T00:22:28.679687+0000","last_became_active":"2024-09-11T00:21:28.230956+0000","last_became_peered":"2024-09-11T00:21:28.230956+0000","last_unstale":"2024-09-11T00:22:28.679687+0000","last_undegraded":"2024-09-11T00:22:28.679687+0000","last_fullsized":"2024-09-11T00:22:28.679687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:28.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865489+0000","last_change":"2024-09-11T00:20:24.023068+0000","last_active":"2024-09-11T00:22:28.865489+0000","last_peered":"2024-09-11T00:22:28.865489+0000","last_clean":"2024-09-11T00:22:28.865489+0000","last_became_active":"2024-09-11T00:20:24.022718+0000","last_became_peered":"2024-09-11T00:20:24.022718+0000","last_unstale":"2024-09-11T00:22:28.865489+0000","last_undegraded":"2024-09-11T00:22:28.865489+0000","last_fullsized":"2024-09-11T00:22:28.865489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.651536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863633+0000","last_change":"2024-09-11T00:21:28.231887+0000","last_active":"2024-09-11T00:22:28.863633+0000","last_peered":"2024-09-11T00:22:28.863633+0000","last_clean":"2024-09-11T00:22:28.863633+0000","last_became_active":"2024-09-11T00:21:28.231456+0000","last_became_peered":"2024-09-11T00:21:28.231456+0000","last_unstale":"2024-09-11T00:22:28.863633+0000","last_undegraded":"2024-09-11T00:22:28.863633+0000","last_fullsized":"2024-09-11T00:22:28.863633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:32.712598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865591+0000","last_change":"2024-09-11T00:20:27.065109+0000","last_active":"2024-09-11T00:22:28.865591+0000","last_peered":"2024-09-11T00:22:28.865591+0000","last_clean":"2024-09-11T00:22:28.865591+0000","last_became_active":"2024-09-11T00:20:27.064677+0000","last_became_peered":"2024-09-11T00:20:27.064677+0000","last_unstale":"2024-09-11T00:22:28.865591+0000","last_undegraded":"2024-09-11T00:22:28.865591+0000","last_fullsized":"2024-09-11T00:22:28.865591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:19.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:32.318753+0000","last_change":"2024-09-11T00:20:24.025347+0000","last_active":"2024-09-11T00:22:32.318753+0000","last_peered":"2024-09-11T00:22:32.318753+0000","last_clean":"2024-09-11T00:22:32.318753+0000","last_became_active":"2024-09-11T00:20:24.024980+0000","last_became_peered":"2024-09-11T00:20:24.024980+0000","last_unstale":"2024-09-11T00:22:32.318753+0000","last_undegraded":"2024-09-11T00:22:32.318753+0000","last_fullsized":"2024-09-11T00:22:32.318753+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:12.530339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863535+0000","last_change":"2024-09-11T00:20:37.440340+0000","last_active":"2024-09-11T00:22:28.863535+0000","last_peered":"2024-09-11T00:22:28.863535+0000","last_clean":"2024-09-11T00:22:28.863535+0000","last_became_active":"2024-09-11T00:20:37.439896+0000","last_became_peered":"2024-09-11T00:20:37.439896+0000","last_unstale":"2024-09-11T00:22:28.863535+0000","last_undegraded":"2024-09-11T00:22:28.863535+0000","last_fullsized":"2024-09-11T00:22:28.863535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:38.774065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679146+0000","last_change":"2024-09-11T00:20:37.445860+0000","last_active":"2024-09-11T00:22:28.679146+0000","last_peered":"2024-09-11T00:22:28.679146+0000","last_clean":"2024-09-11T00:22:28.679146+0000","last_became_active":"2024-09-11T00:20:37.445450+0000","last_became_peered":"2024-09-11T00:20:37.445450+0000","last_unstale":"2024-09-11T00:22:28.679146+0000","last_undegraded":"2024-09-11T00:22:28.679146+0000","last_fullsized":"2024-09-11T00:22:28.679146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:44.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680379+0000","last_change":"2024-09-11T00:20:27.073529+0000","last_active":"2024-09-11T00:22:28.680379+0000","last_peered":"2024-09-11T00:22:28.680379+0000","last_clean":"2024-09-11T00:22:28.680379+0000","last_became_active":"2024-09-11T00:20:27.073168+0000","last_became_peered":"2024-09-11T00:20:27.073168+0000","last_unstale":"2024-09-11T00:22:28.680379+0000","last_undegraded":"2024-09-11T00:22:28.680379+0000","last_fullsized":"2024-09-11T00:22:28.680379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:16.407543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734057+0000","last_change":"2024-09-11T00:20:24.021778+0000","last_active":"2024-09-11T00:22:28.734057+0000","last_peered":"2024-09-11T00:22:28.734057+0000","last_clean":"2024-09-11T00:22:28.734057+0000","last_became_active":"2024-09-11T00:20:24.021443+0000","last_became_peered":"2024-09-11T00:20:24.021443+0000","last_unstale":"2024-09-11T00:22:28.734057+0000","last_undegraded":"2024-09-11T00:22:28.734057+0000","last_fullsized":"2024-09-11T00:22:28.734057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864345+0000","last_change":"2024-09-11T00:20:37.444279+0000","last_active":"2024-09-11T00:22:28.864345+0000","last_peered":"2024-09-11T00:22:28.864345+0000","last_clean":"2024-09-11T00:22:28.864345+0000","last_became_active":"2024-09-11T00:20:37.443821+0000","last_became_peered":"2024-09-11T00:20:37.443821+0000","last_unstale":"2024-09-11T00:22:28.864345+0000","last_undegraded":"2024-09-11T00:22:28.864345+0000","last_fullsized":"2024-09-11T00:22:28.864345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:10.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863544+0000","last_change":"2024-09-11T00:20:27.066155+0000","last_active":"2024-09-11T00:22:28.863544+0000","last_peered":"2024-09-11T00:22:28.863544+0000","last_clean":"2024-09-11T00:22:28.863544+0000","last_became_active":"2024-09-11T00:20:27.057262+0000","last_became_peered":"2024-09-11T00:20:27.057262+0000","last_unstale":"2024-09-11T00:22:28.863544+0000","last_undegraded":"2024-09-11T00:22:28.863544+0000","last_fullsized":"2024-09-11T00:22:28.863544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:54.160215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682589+0000","last_change":"2024-09-11T00:20:24.026123+0000","last_active":"2024-09-11T00:22:28.682589+0000","last_peered":"2024-09-11T00:22:28.682589+0000","last_clean":"2024-09-11T00:22:28.682589+0000","last_became_active":"2024-09-11T00:20:24.025832+0000","last_became_peered":"2024-09-11T00:20:24.025832+0000","last_unstale":"2024-09-11T00:22:28.682589+0000","last_undegraded":"2024-09-11T00:22:28.682589+0000","last_fullsized":"2024-09-11T00:22:28.682589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:45.444840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830114+0000","last_change":"2024-09-11T00:20:24.028523+0000","last_active":"2024-09-11T00:22:28.830114+0000","last_peered":"2024-09-11T00:22:28.830114+0000","last_clean":"2024-09-11T00:22:28.830114+0000","last_became_active":"2024-09-11T00:20:24.027801+0000","last_became_peered":"2024-09-11T00:20:24.027801+0000","last_unstale":"2024-09-11T00:22:28.830114+0000","last_undegraded":"2024-09-11T00:22:28.830114+0000","last_fullsized":"2024-09-11T00:22:28.830114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:14.174983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734264+0000","last_change":"2024-09-11T00:20:27.074634+0000","last_active":"2024-09-11T00:22:28.734264+0000","last_peered":"2024-09-11T00:22:28.734264+0000","last_clean":"2024-09-11T00:22:28.734264+0000","last_became_active":"2024-09-11T00:20:27.074292+0000","last_became_peered":"2024-09-11T00:20:27.074292+0000","last_unstale":"2024-09-11T00:22:28.734264+0000","last_undegraded":"2024-09-11T00:22:28.734264+0000","last_fullsized":"2024-09-11T00:22:28.734264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:07.467262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679366+0000","last_change":"2024-09-11T00:20:37.454341+0000","last_active":"2024-09-11T00:22:28.679366+0000","last_peered":"2024-09-11T00:22:28.679366+0000","last_clean":"2024-09-11T00:22:28.679366+0000","last_became_active":"2024-09-11T00:20:37.453868+0000","last_became_peered":"2024-09-11T00:20:37.453868+0000","last_unstale":"2024-09-11T00:22:28.679366+0000","last_undegraded":"2024-09-11T00:22:28.679366+0000","last_fullsized":"2024-09-11T00:22:28.679366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.015338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:32.319508+0000","last_change":"2024-09-11T00:20:24.025184+0000","last_active":"2024-09-11T00:22:32.319508+0000","last_peered":"2024-09-11T00:22:32.319508+0000","last_clean":"2024-09-11T00:22:32.319508+0000","last_became_active":"2024-09-11T00:20:24.024670+0000","last_became_peered":"2024-09-11T00:20:24.024670+0000","last_unstale":"2024-09-11T00:22:32.319508+0000","last_undegraded":"2024-09-11T00:22:32.319508+0000","last_fullsized":"2024-09-11T00:22:32.319508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:40.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864184+0000","last_change":"2024-09-11T00:20:27.057822+0000","last_active":"2024-09-11T00:22:28.864184+0000","last_peered":"2024-09-11T00:22:28.864184+0000","last_clean":"2024-09-11T00:22:28.864184+0000","last_became_active":"2024-09-11T00:20:27.057518+0000","last_became_peered":"2024-09-11T00:20:27.057518+0000","last_unstale":"2024-09-11T00:22:28.864184+0000","last_undegraded":"2024-09-11T00:22:28.864184+0000","last_fullsized":"2024-09-11T00:22:28.864184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:00.264327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679470+0000","last_change":"2024-09-11T00:20:37.438369+0000","last_active":"2024-09-11T00:22:28.679470+0000","last_peered":"2024-09-11T00:22:28.679470+0000","last_clean":"2024-09-11T00:22:28.679470+0000","last_became_active":"2024-09-11T00:20:37.437942+0000","last_became_peered":"2024-09-11T00:20:37.437942+0000","last_unstale":"2024-09-11T00:22:28.679470+0000","last_undegraded":"2024-09-11T00:22:28.679470+0000","last_fullsized":"2024-09-11T00:22:28.679470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.410905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863977+0000","last_change":"2024-09-11T00:20:27.057449+0000","last_active":"2024-09-11T00:22:28.863977+0000","last_peered":"2024-09-11T00:22:28.863977+0000","last_clean":"2024-09-11T00:22:28.863977+0000","last_became_active":"2024-09-11T00:20:27.057146+0000","last_became_peered":"2024-09-11T00:20:27.057146+0000","last_unstale":"2024-09-11T00:22:28.863977+0000","last_undegraded":"2024-09-11T00:22:28.863977+0000","last_fullsized":"2024-09-11T00:22:28.863977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:57.895062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679881+0000","last_change":"2024-09-11T00:20:27.061515+0000","last_active":"2024-09-11T00:22:28.679881+0000","last_peered":"2024-09-11T00:22:28.679881+0000","last_clean":"2024-09-11T00:22:28.679881+0000","last_became_active":"2024-09-11T00:20:27.061175+0000","last_became_peered":"2024-09-11T00:20:27.061175+0000","last_unstale":"2024-09-11T00:22:28.679881+0000","last_undegraded":"2024-09-11T00:22:28.679881+0000","last_fullsized":"2024-09-11T00:22:28.679881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:03.112574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679377+0000","last_change":"2024-09-11T00:20:27.063015+0000","last_active":"2024-09-11T00:22:28.679377+0000","last_peered":"2024-09-11T00:22:28.679377+0000","last_clean":"2024-09-11T00:22:28.679377+0000","last_became_active":"2024-09-11T00:20:27.062689+0000","last_became_peered":"2024-09-11T00:20:27.062689+0000","last_unstale":"2024-09-11T00:22:28.679377+0000","last_undegraded":"2024-09-11T00:22:28.679377+0000","last_fullsized":"2024-09-11T00:22:28.679377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.736073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679155+0000","last_change":"2024-09-11T00:20:27.062196+0000","last_active":"2024-09-11T00:22:28.679155+0000","last_peered":"2024-09-11T00:22:28.679155+0000","last_clean":"2024-09-11T00:22:28.679155+0000","last_became_active":"2024-09-11T00:20:27.061918+0000","last_became_peered":"2024-09-11T00:20:27.061918+0000","last_unstale":"2024-09-11T00:22:28.679155+0000","last_undegraded":"2024-09-11T00:22:28.679155+0000","last_fullsized":"2024-09-11T00:22:28.679155+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:45.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682933+0000","last_change":"2024-09-11T00:20:27.059021+0000","last_active":"2024-09-11T00:22:28.682933+0000","last_peered":"2024-09-11T00:22:28.682933+0000","last_clean":"2024-09-11T00:22:28.682933+0000","last_became_active":"2024-09-11T00:20:27.058726+0000","last_became_peered":"2024-09-11T00:20:27.058726+0000","last_unstale":"2024-09-11T00:22:28.682933+0000","last_undegraded":"2024-09-11T00:22:28.682933+0000","last_fullsized":"2024-09-11T00:22:28.682933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:23.900236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864216+0000","last_change":"2024-09-11T00:20:27.057593+0000","last_active":"2024-09-11T00:22:28.864216+0000","last_peered":"2024-09-11T00:22:28.864216+0000","last_clean":"2024-09-11T00:22:28.864216+0000","last_became_active":"2024-09-11T00:20:27.057310+0000","last_became_peered":"2024-09-11T00:20:27.057310+0000","last_unstale":"2024-09-11T00:22:28.864216+0000","last_undegraded":"2024-09-11T00:22:28.864216+0000","last_fullsized":"2024-09-11T00:22:28.864216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:52.355218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679785+0000","last_change":"2024-09-11T00:20:27.061140+0000","last_active":"2024-09-11T00:22:28.679785+0000","last_peered":"2024-09-11T00:22:28.679785+0000","last_clean":"2024-09-11T00:22:28.679785+0000","last_became_active":"2024-09-11T00:20:27.060500+0000","last_became_peered":"2024-09-11T00:20:27.060500+0000","last_unstale":"2024-09-11T00:22:28.679785+0000","last_undegraded":"2024-09-11T00:22:28.679785+0000","last_fullsized":"2024-09-11T00:22:28.679785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.159068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863940+0000","last_change":"2024-09-11T00:20:27.061936+0000","last_active":"2024-09-11T00:22:28.863940+0000","last_peered":"2024-09-11T00:22:28.863940+0000","last_clean":"2024-09-11T00:22:28.863940+0000","last_became_active":"2024-09-11T00:20:27.061616+0000","last_became_peered":"2024-09-11T00:20:27.061616+0000","last_unstale":"2024-09-11T00:22:28.863940+0000","last_undegraded":"2024-09-11T00:22:28.863940+0000","last_fullsized":"2024-09-11T00:22:28.863940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:32.185634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733947+0000","last_change":"2024-09-11T00:20:27.074645+0000","last_active":"2024-09-11T00:22:28.733947+0000","last_peered":"2024-09-11T00:22:28.733947+0000","last_clean":"2024-09-11T00:22:28.733947+0000","last_became_active":"2024-09-11T00:20:27.074355+0000","last_became_peered":"2024-09-11T00:20:27.074355+0000","last_unstale":"2024-09-11T00:22:28.733947+0000","last_undegraded":"2024-09-11T00:22:28.733947+0000","last_fullsized":"2024-09-11T00:22:28.733947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863600+0000","last_change":"2024-09-11T00:20:27.069588+0000","last_active":"2024-09-11T00:22:28.863600+0000","last_peered":"2024-09-11T00:22:28.863600+0000","last_clean":"2024-09-11T00:22:28.863600+0000","last_became_active":"2024-09-11T00:20:27.067403+0000","last_became_peered":"2024-09-11T00:20:27.067403+0000","last_unstale":"2024-09-11T00:22:28.863600+0000","last_undegraded":"2024-09-11T00:22:28.863600+0000","last_fullsized":"2024-09-11T00:22:28.863600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:29.439013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865193+0000","last_change":"2024-09-11T00:20:27.071586+0000","last_active":"2024-09-11T00:22:28.865193+0000","last_peered":"2024-09-11T00:22:28.865193+0000","last_clean":"2024-09-11T00:22:28.865193+0000","last_became_active":"2024-09-11T00:20:27.071072+0000","last_became_peered":"2024-09-11T00:20:27.071072+0000","last_unstale":"2024-09-11T00:22:28.865193+0000","last_undegraded":"2024-09-11T00:22:28.865193+0000","last_fullsized":"2024-09-11T00:22:28.865193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:34.133027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683353+0000","last_change":"2024-09-11T00:20:27.066683+0000","last_active":"2024-09-11T00:22:28.683353+0000","last_peered":"2024-09-11T00:22:28.683353+0000","last_clean":"2024-09-11T00:22:28.683353+0000","last_became_active":"2024-09-11T00:20:27.066355+0000","last_became_peered":"2024-09-11T00:20:27.066355+0000","last_unstale":"2024-09-11T00:22:28.683353+0000","last_undegraded":"2024-09-11T00:22:28.683353+0000","last_fullsized":"2024-09-11T00:22:28.683353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:26.868072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864043+0000","last_change":"2024-09-11T00:20:27.060546+0000","last_active":"2024-09-11T00:22:28.864043+0000","last_peered":"2024-09-11T00:22:28.864043+0000","last_clean":"2024-09-11T00:22:28.864043+0000","last_became_active":"2024-09-11T00:20:27.059619+0000","last_became_peered":"2024-09-11T00:20:27.059619+0000","last_unstale":"2024-09-11T00:22:28.864043+0000","last_undegraded":"2024-09-11T00:22:28.864043+0000","last_fullsized":"2024-09-11T00:22:28.864043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.319854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679643+0000","last_change":"2024-09-11T00:20:27.073342+0000","last_active":"2024-09-11T00:22:28.679643+0000","last_peered":"2024-09-11T00:22:28.679643+0000","last_clean":"2024-09-11T00:22:28.679643+0000","last_became_active":"2024-09-11T00:20:27.072435+0000","last_became_peered":"2024-09-11T00:20:27.072435+0000","last_unstale":"2024-09-11T00:22:28.679643+0000","last_undegraded":"2024-09-11T00:22:28.679643+0000","last_fullsized":"2024-09-11T00:22:28.679643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:15.943515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680856+0000","last_change":"2024-09-11T00:20:27.063220+0000","last_active":"2024-09-11T00:22:28.680856+0000","last_peered":"2024-09-11T00:22:28.680856+0000","last_clean":"2024-09-11T00:22:28.680856+0000","last_became_active":"2024-09-11T00:20:27.062823+0000","last_became_peered":"2024-09-11T00:20:27.062823+0000","last_unstale":"2024-09-11T00:22:28.680856+0000","last_undegraded":"2024-09-11T00:22:28.680856+0000","last_fullsized":"2024-09-11T00:22:28.680856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:22.430295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864501+0000","last_change":"2024-09-11T00:20:27.062088+0000","last_active":"2024-09-11T00:22:28.864501+0000","last_peered":"2024-09-11T00:22:28.864501+0000","last_clean":"2024-09-11T00:22:28.864501+0000","last_became_active":"2024-09-11T00:20:27.061639+0000","last_became_peered":"2024-09-11T00:20:27.061639+0000","last_unstale":"2024-09-11T00:22:28.864501+0000","last_undegraded":"2024-09-11T00:22:28.864501+0000","last_fullsized":"2024-09-11T00:22:28.864501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.965908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863958+0000","last_change":"2024-09-11T00:20:27.067170+0000","last_active":"2024-09-11T00:22:28.863958+0000","last_peered":"2024-09-11T00:22:28.863958+0000","last_clean":"2024-09-11T00:22:28.863958+0000","last_became_active":"2024-09-11T00:20:27.066851+0000","last_became_peered":"2024-09-11T00:20:27.066851+0000","last_unstale":"2024-09-11T00:22:28.863958+0000","last_undegraded":"2024-09-11T00:22:28.863958+0000","last_fullsized":"2024-09-11T00:22:28.863958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:51.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.678866+0000","last_change":"2024-09-11T00:21:28.233390+0000","last_active":"2024-09-11T00:22:28.678866+0000","last_peered":"2024-09-11T00:22:28.678866+0000","last_clean":"2024-09-11T00:22:28.678866+0000","last_became_active":"2024-09-11T00:21:28.232886+0000","last_became_peered":"2024-09-11T00:21:28.232886+0000","last_unstale":"2024-09-11T00:22:28.678866+0000","last_undegraded":"2024-09-11T00:22:28.678866+0000","last_fullsized":"2024-09-11T00:22:28.678866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:26.702069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.678865+0000","last_change":"2024-09-11T00:20:27.059553+0000","last_active":"2024-09-11T00:22:28.678865+0000","last_peered":"2024-09-11T00:22:28.678865+0000","last_clean":"2024-09-11T00:22:28.678865+0000","last_became_active":"2024-09-11T00:20:27.058979+0000","last_became_peered":"2024-09-11T00:20:27.058979+0000","last_unstale":"2024-09-11T00:22:28.678865+0000","last_undegraded":"2024-09-11T00:22:28.678865+0000","last_fullsized":"2024-09-11T00:22:28.678865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:21.472828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.831629+0000","last_change":"2024-09-11T00:22:28.831536+0000","last_active":"2024-09-11T00:22:28.831629+0000","last_peered":"2024-09-11T00:22:28.831629+0000","last_clean":"2024-09-11T00:22:28.831629+0000","last_became_active":"2024-09-11T00:22:28.831151+0000","last_became_peered":"2024-09-11T00:22:28.831151+0000","last_unstale":"2024-09-11T00:22:28.831629+0000","last_undegraded":"2024-09-11T00:22:28.831629+0000","last_fullsized":"2024-09-11T00:22:28.831629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.532570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864880+0000","last_change":"2024-09-11T00:20:27.059140+0000","last_active":"2024-09-11T00:22:28.864880+0000","last_peered":"2024-09-11T00:22:28.864880+0000","last_clean":"2024-09-11T00:22:28.864880+0000","last_became_active":"2024-09-11T00:20:27.058765+0000","last_became_peered":"2024-09-11T00:20:27.058765+0000","last_unstale":"2024-09-11T00:22:28.864880+0000","last_undegraded":"2024-09-11T00:22:28.864880+0000","last_fullsized":"2024-09-11T00:22:28.864880+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:43.947411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866106+0000","last_change":"2024-09-11T00:20:27.061149+0000","last_active":"2024-09-11T00:22:28.866106+0000","last_peered":"2024-09-11T00:22:28.866106+0000","last_clean":"2024-09-11T00:22:28.866106+0000","last_became_active":"2024-09-11T00:20:27.059949+0000","last_became_peered":"2024-09-11T00:20:27.059949+0000","last_unstale":"2024-09-11T00:22:28.866106+0000","last_undegraded":"2024-09-11T00:22:28.866106+0000","last_fullsized":"2024-09-11T00:22:28.866106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:03.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830617+0000","last_change":"2024-09-11T00:21:28.231322+0000","last_active":"2024-09-11T00:22:28.830617+0000","last_peered":"2024-09-11T00:22:28.830617+0000","last_clean":"2024-09-11T00:22:28.830617+0000","last_became_active":"2024-09-11T00:21:28.231075+0000","last_became_peered":"2024-09-11T00:21:28.231075+0000","last_unstale":"2024-09-11T00:22:28.830617+0000","last_undegraded":"2024-09-11T00:22:28.830617+0000","last_fullsized":"2024-09-11T00:22:28.830617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:50.887001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865942+0000","last_change":"2024-09-11T00:20:27.058175+0000","last_active":"2024-09-11T00:22:28.865942+0000","last_peered":"2024-09-11T00:22:28.865942+0000","last_clean":"2024-09-11T00:22:28.865942+0000","last_became_active":"2024-09-11T00:20:27.057797+0000","last_became_peered":"2024-09-11T00:20:27.057797+0000","last_unstale":"2024-09-11T00:22:28.865942+0000","last_undegraded":"2024-09-11T00:22:28.865942+0000","last_fullsized":"2024-09-11T00:22:28.865942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:29.370400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863468+0000","last_change":"2024-09-11T00:20:27.062024+0000","last_active":"2024-09-11T00:22:28.863468+0000","last_peered":"2024-09-11T00:22:28.863468+0000","last_clean":"2024-09-11T00:22:28.863468+0000","last_became_active":"2024-09-11T00:20:27.061763+0000","last_became_peered":"2024-09-11T00:20:27.061763+0000","last_unstale":"2024-09-11T00:22:28.863468+0000","last_undegraded":"2024-09-11T00:22:28.863468+0000","last_fullsized":"2024-09-11T00:22:28.863468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:07.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865489+0000","last_change":"2024-09-11T00:20:27.060258+0000","last_active":"2024-09-11T00:22:28.865489+0000","last_peered":"2024-09-11T00:22:28.865489+0000","last_clean":"2024-09-11T00:22:28.865489+0000","last_became_active":"2024-09-11T00:20:27.059475+0000","last_became_peered":"2024-09-11T00:20:27.059475+0000","last_unstale":"2024-09-11T00:22:28.865489+0000","last_undegraded":"2024-09-11T00:22:28.865489+0000","last_fullsized":"2024-09-11T00:22:28.865489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:27.186322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830107+0000","last_change":"2024-09-11T00:20:27.059730+0000","last_active":"2024-09-11T00:22:28.830107+0000","last_peered":"2024-09-11T00:22:28.830107+0000","last_clean":"2024-09-11T00:22:28.830107+0000","last_became_active":"2024-09-11T00:20:27.059389+0000","last_became_peered":"2024-09-11T00:20:27.059389+0000","last_unstale":"2024-09-11T00:22:28.830107+0000","last_undegraded":"2024-09-11T00:22:28.830107+0000","last_fullsized":"2024-09-11T00:22:28.830107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864185+0000","last_change":"2024-09-11T00:20:27.072765+0000","last_active":"2024-09-11T00:22:28.864185+0000","last_peered":"2024-09-11T00:22:28.864185+0000","last_clean":"2024-09-11T00:22:28.864185+0000","last_became_active":"2024-09-11T00:20:27.072392+0000","last_became_peered":"2024-09-11T00:20:27.072392+0000","last_unstale":"2024-09-11T00:22:28.864185+0000","last_undegraded":"2024-09-11T00:22:28.864185+0000","last_fullsized":"2024-09-11T00:22:28.864185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:27.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864725+0000","last_change":"2024-09-11T00:20:27.056575+0000","last_active":"2024-09-11T00:22:28.864725+0000","last_peered":"2024-09-11T00:22:28.864725+0000","last_clean":"2024-09-11T00:22:28.864725+0000","last_became_active":"2024-09-11T00:20:27.056264+0000","last_became_peered":"2024-09-11T00:20:27.056264+0000","last_unstale":"2024-09-11T00:22:28.864725+0000","last_undegraded":"2024-09-11T00:22:28.864725+0000","last_fullsized":"2024-09-11T00:22:28.864725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:39.722518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679344+0000","last_change":"2024-09-11T00:20:27.065153+0000","last_active":"2024-09-11T00:22:28.679344+0000","last_peered":"2024-09-11T00:22:28.679344+0000","last_clean":"2024-09-11T00:22:28.679344+0000","last_became_active":"2024-09-11T00:20:27.062978+0000","last_became_peered":"2024-09-11T00:20:27.062978+0000","last_unstale":"2024-09-11T00:22:28.679344+0000","last_undegraded":"2024-09-11T00:22:28.679344+0000","last_fullsized":"2024-09-11T00:22:28.679344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:37.820467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679252+0000","last_change":"2024-09-11T00:20:27.070085+0000","last_active":"2024-09-11T00:22:28.679252+0000","last_peered":"2024-09-11T00:22:28.679252+0000","last_clean":"2024-09-11T00:22:28.679252+0000","last_became_active":"2024-09-11T00:20:27.069809+0000","last_became_peered":"2024-09-11T00:20:27.069809+0000","last_unstale":"2024-09-11T00:22:28.679252+0000","last_undegraded":"2024-09-11T00:22:28.679252+0000","last_fullsized":"2024-09-11T00:22:28.679252+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:01.048590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864471+0000","last_change":"2024-09-11T00:20:27.064584+0000","last_active":"2024-09-11T00:22:28.864471+0000","last_peered":"2024-09-11T00:22:28.864471+0000","last_clean":"2024-09-11T00:22:28.864471+0000","last_became_active":"2024-09-11T00:20:27.064271+0000","last_became_peered":"2024-09-11T00:20:27.064271+0000","last_unstale":"2024-09-11T00:22:28.864471+0000","last_undegraded":"2024-09-11T00:22:28.864471+0000","last_fullsized":"2024-09-11T00:22:28.864471+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:53.717744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":438,"num_read_kb":287,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710759,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1015808,"data_stored":505752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906988,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1712128,"data_stored":1164346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070514,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":519338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266744,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":515050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1634304,"data_stored":1111071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":31914445},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":510418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1642496,"data_stored":1106659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":515068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":510612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1110016,"data_stored":574141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":572400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":511806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:22:36.897 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:22:36.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:36 smithi117 ceph-mon[35525]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:22:36.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:36 smithi117 ceph-mon[35525]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-09-11T00:22:37.295 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:37.912 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 -- 172.21.15.103:0/2207125128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c10a9d0 msgr2=0x7f4b6c10ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 --2- 172.21.15.103:0/2207125128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c10ae10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009860 tx=0x7f4b5802a340 comp rx=0 tx=0).stop 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 -- 172.21.15.103:0/2207125128 shutdown_connections 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 --2- 172.21.15.103:0/2207125128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b6c10b350 0x7f4b6c111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 --2- 172.21.15.103:0/2207125128 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c10ae10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 --2- 172.21.15.103:0/2207125128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c104fc0 0x7f4b6c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 -- 172.21.15.103:0/2207125128 >> 172.21.15.103:0/2207125128 conn(0x7f4b6c1009f0 msgr2=0x7f4b6c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 -- 172.21.15.103:0/2207125128 shutdown_connections 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.891+0000 7f4b71000640 1 -- 172.21.15.103:0/2207125128 wait complete. 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.892+0000 7f4b71000640 1 Processor -- start 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.892+0000 7f4b71000640 1 -- start start 2024-09-11T00:22:37.913 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.892+0000 7f4b71000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c104fc0 0x7f4b6c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.892+0000 7f4b71000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.892+0000 7f4b71000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b6c10b350 0x7f4b6c1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b71000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c114f50 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b71000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c114dd0 con 0x7f4b6c10b350 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b71000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6c1150d0 con 0x7f4b6c104fc0 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40988/0 (socket says 172.21.15.103:40988) 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 -- 172.21.15.103:0/4152292821 learned_addr learned my addr 172.21.15.103:0/4152292821 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b6a800640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b6c10b350 0x7f4b6c1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69e00640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c104fc0 0x7f4b6c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 -- 172.21.15.103:0/4152292821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c104fc0 msgr2=0x7f4b6c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c104fc0 0x7f4b6c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:37.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 -- 172.21.15.103:0/4152292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b6c10b350 msgr2=0x7f4b6c1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b6c10b350 0x7f4b6c1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b69400640 1 -- 172.21.15.103:0/4152292821 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6c1a14f0 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.893+0000 7f4b6a800640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b6c10b350 0x7f4b6c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.894+0000 7f4b69400640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c19b980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4b58035040 tx=0x7f4b58032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.894+0000 7f4b62a00640 1 -- 172.21.15.103:0/4152292821 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58037040 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.894+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6c1a1780 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.894+0000 7f4b62a00640 1 -- 172.21.15.103:0/4152292821 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b58041070 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.894+0000 7f4b62a00640 1 -- 172.21.15.103:0/4152292821 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b58003850 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.894+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b6c1a7ea0 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.895+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6c1053a0 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.897+0000 7f4b62a00640 1 -- 172.21.15.103:0/4152292821 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f4b580039f0 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.899+0000 7f4b62a00640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4b500b5370 0x7f4b500b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:37.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.899+0000 7f4b62a00640 1 -- 172.21.15.103:0/4152292821 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f4b58105b00 con 0x7f4b6c10a9d0 2024-09-11T00:22:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.899+0000 7f4b69e00640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4b500b5370 0x7f4b500b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.900+0000 7f4b69e00640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4b500b5370 0x7f4b500b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009530 tx=0x7f4b5c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:37.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:37.903+0000 7f4b62a00640 1 -- 172.21.15.103:0/4152292821 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4b580cc470 con 0x7f4b6c10a9d0 2024-09-11T00:22:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.122+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b6c10d160 con 0x7f4b500b5370 2024-09-11T00:22:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.128+0000 7f4b62a00640 1 -- 172.21.15.103:0/4152292821 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340358 (secure 0 0 0) 0x7f4b6c10d160 con 0x7f4b500b5370 2024-09-11T00:22:38.134 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:22:38.135 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-11T00:22:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4b500b5370 msgr2=0x7f4b500b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4b500b5370 0x7f4b500b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c009530 tx=0x7f4b5c009260 comp rx=0 tx=0).stop 2024-09-11T00:22:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c10a9d0 msgr2=0x7f4b6c19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c19b980 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4b58035040 tx=0x7f4b58032d50 comp rx=0 tx=0).stop 2024-09-11T00:22:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 shutdown_connections 2024-09-11T00:22:38.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4b500b5370 0x7f4b500b7830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b6c10b350 0x7f4b6c1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4b6c10a9d0 0x7f4b6c19b980 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 --2- 172.21.15.103:0/4152292821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b6c104fc0 0x7f4b6c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 >> 172.21.15.103:0/4152292821 conn(0x7f4b6c1009f0 msgr2=0x7f4b6c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 shutdown_connections 2024-09-11T00:22:38.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:38.133+0000 7f4b71000640 1 -- 172.21.15.103:0/4152292821 wait complete. 2024-09-11T00:22:38.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:22:38.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:22:38.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:38 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:22:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:22:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:38 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:22:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:22:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:38 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:22:38.973 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-11T00:22:36.392091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656675,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":519,"num_read_kb":356,"num_write":294,"num_write_kb":2269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375904,"kb_used_data":14084,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565920,"statfs":{"total":1151940427776,"available":1151555502080,"internally_reserved":0,"allocated":14422016,"data_stored":8126027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.718508"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680924+0000","last_change":"2024-09-11T00:20:24.027578+0000","last_active":"2024-09-11T00:22:28.680924+0000","last_peered":"2024-09-11T00:22:28.680924+0000","last_clean":"2024-09-11T00:22:28.680924+0000","last_became_active":"2024-09-11T00:20:24.027174+0000","last_became_peered":"2024-09-11T00:20:24.027174+0000","last_unstale":"2024-09-11T00:22:28.680924+0000","last_undegraded":"2024-09-11T00:22:28.680924+0000","last_fullsized":"2024-09-11T00:22:28.680924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:15.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863777+0000","last_change":"2024-09-11T00:20:27.064047+0000","last_active":"2024-09-11T00:22:28.863777+0000","last_peered":"2024-09-11T00:22:28.863777+0000","last_clean":"2024-09-11T00:22:28.863777+0000","last_became_active":"2024-09-11T00:20:27.063756+0000","last_became_peered":"2024-09-11T00:20:27.063756+0000","last_unstale":"2024-09-11T00:22:28.863777+0000","last_undegraded":"2024-09-11T00:22:28.863777+0000","last_fullsized":"2024-09-11T00:22:28.863777+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.695888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679187+0000","last_change":"2024-09-11T00:20:37.449428+0000","last_active":"2024-09-11T00:22:28.679187+0000","last_peered":"2024-09-11T00:22:28.679187+0000","last_clean":"2024-09-11T00:22:28.679187+0000","last_became_active":"2024-09-11T00:20:37.449015+0000","last_became_peered":"2024-09-11T00:20:37.449015+0000","last_unstale":"2024-09-11T00:22:28.679187+0000","last_undegraded":"2024-09-11T00:22:28.679187+0000","last_fullsized":"2024-09-11T00:22:28.679187+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:42.983990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864076+0000","last_change":"2024-09-11T00:20:24.035966+0000","last_active":"2024-09-11T00:22:28.864076+0000","last_peered":"2024-09-11T00:22:28.864076+0000","last_clean":"2024-09-11T00:22:28.864076+0000","last_became_active":"2024-09-11T00:20:24.035663+0000","last_became_peered":"2024-09-11T00:20:24.035663+0000","last_unstale":"2024-09-11T00:22:28.864076+0000","last_undegraded":"2024-09-11T00:22:28.864076+0000","last_fullsized":"2024-09-11T00:22:28.864076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:22.529392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679684+0000","last_change":"2024-09-11T00:20:27.066470+0000","last_active":"2024-09-11T00:22:28.679684+0000","last_peered":"2024-09-11T00:22:28.679684+0000","last_clean":"2024-09-11T00:22:28.679684+0000","last_became_active":"2024-09-11T00:20:27.065424+0000","last_became_peered":"2024-09-11T00:20:27.065424+0000","last_unstale":"2024-09-11T00:22:28.679684+0000","last_undegraded":"2024-09-11T00:22:28.679684+0000","last_fullsized":"2024-09-11T00:22:28.679684+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:39.967332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864756+0000","last_change":"2024-09-11T00:20:37.453105+0000","last_active":"2024-09-11T00:22:28.864756+0000","last_peered":"2024-09-11T00:22:28.864756+0000","last_clean":"2024-09-11T00:22:28.864756+0000","last_became_active":"2024-09-11T00:20:37.452319+0000","last_became_peered":"2024-09-11T00:20:37.452319+0000","last_unstale":"2024-09-11T00:22:28.864756+0000","last_undegraded":"2024-09-11T00:22:28.864756+0000","last_fullsized":"2024-09-11T00:22:28.864756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:05.873046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864760+0000","last_change":"2024-09-11T00:20:24.022277+0000","last_active":"2024-09-11T00:22:28.864760+0000","last_peered":"2024-09-11T00:22:28.864760+0000","last_clean":"2024-09-11T00:22:28.864760+0000","last_became_active":"2024-09-11T00:20:24.021893+0000","last_became_peered":"2024-09-11T00:20:24.021893+0000","last_unstale":"2024-09-11T00:22:28.864760+0000","last_undegraded":"2024-09-11T00:22:28.864760+0000","last_fullsized":"2024-09-11T00:22:28.864760+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.994032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864847+0000","last_change":"2024-09-11T00:20:27.063368+0000","last_active":"2024-09-11T00:22:28.864847+0000","last_peered":"2024-09-11T00:22:28.864847+0000","last_clean":"2024-09-11T00:22:28.864847+0000","last_became_active":"2024-09-11T00:20:27.063051+0000","last_became_peered":"2024-09-11T00:20:27.063051+0000","last_unstale":"2024-09-11T00:22:28.864847+0000","last_undegraded":"2024-09-11T00:22:28.864847+0000","last_fullsized":"2024-09-11T00:22:28.864847+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:09.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864787+0000","last_change":"2024-09-11T00:21:28.229073+0000","last_active":"2024-09-11T00:22:28.864787+0000","last_peered":"2024-09-11T00:22:28.864787+0000","last_clean":"2024-09-11T00:22:28.864787+0000","last_became_active":"2024-09-11T00:21:28.228775+0000","last_became_peered":"2024-09-11T00:21:28.228775+0000","last_unstale":"2024-09-11T00:22:28.864787+0000","last_undegraded":"2024-09-11T00:22:28.864787+0000","last_fullsized":"2024-09-11T00:22:28.864787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.886727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864712+0000","last_change":"2024-09-11T00:20:24.022562+0000","last_active":"2024-09-11T00:22:28.864712+0000","last_peered":"2024-09-11T00:22:28.864712+0000","last_clean":"2024-09-11T00:22:28.864712+0000","last_became_active":"2024-09-11T00:20:24.022207+0000","last_became_peered":"2024-09-11T00:20:24.022207+0000","last_unstale":"2024-09-11T00:22:28.864712+0000","last_undegraded":"2024-09-11T00:22:28.864712+0000","last_fullsized":"2024-09-11T00:22:28.864712+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:01.071034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734397+0000","last_change":"2024-09-11T00:20:27.062572+0000","last_active":"2024-09-11T00:22:28.734397+0000","last_peered":"2024-09-11T00:22:28.734397+0000","last_clean":"2024-09-11T00:22:28.734397+0000","last_became_active":"2024-09-11T00:20:27.062288+0000","last_became_peered":"2024-09-11T00:20:27.062288+0000","last_unstale":"2024-09-11T00:22:28.734397+0000","last_undegraded":"2024-09-11T00:22:28.734397+0000","last_fullsized":"2024-09-11T00:22:28.734397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:37.391822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864621+0000","last_change":"2024-09-11T00:20:37.444028+0000","last_active":"2024-09-11T00:22:28.864621+0000","last_peered":"2024-09-11T00:22:28.864621+0000","last_clean":"2024-09-11T00:22:28.864621+0000","last_became_active":"2024-09-11T00:20:37.443344+0000","last_became_peered":"2024-09-11T00:20:37.443344+0000","last_unstale":"2024-09-11T00:22:28.864621+0000","last_undegraded":"2024-09-11T00:22:28.864621+0000","last_fullsized":"2024-09-11T00:22:28.864621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.347054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866013+0000","last_change":"2024-09-11T00:20:27.060317+0000","last_active":"2024-09-11T00:22:28.866013+0000","last_peered":"2024-09-11T00:22:28.866013+0000","last_clean":"2024-09-11T00:22:28.866013+0000","last_became_active":"2024-09-11T00:20:27.059983+0000","last_became_peered":"2024-09-11T00:20:27.059983+0000","last_unstale":"2024-09-11T00:22:28.866013+0000","last_undegraded":"2024-09-11T00:22:28.866013+0000","last_fullsized":"2024-09-11T00:22:28.866013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:28.405553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864185+0000","last_change":"2024-09-11T00:20:24.025555+0000","last_active":"2024-09-11T00:22:28.864185+0000","last_peered":"2024-09-11T00:22:28.864185+0000","last_clean":"2024-09-11T00:22:28.864185+0000","last_became_active":"2024-09-11T00:20:24.025171+0000","last_became_peered":"2024-09-11T00:20:24.025171+0000","last_unstale":"2024-09-11T00:22:28.864185+0000","last_undegraded":"2024-09-11T00:22:28.864185+0000","last_fullsized":"2024-09-11T00:22:28.864185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:22.708985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864433+0000","last_change":"2024-09-11T00:20:37.442057+0000","last_active":"2024-09-11T00:22:28.864433+0000","last_peered":"2024-09-11T00:22:28.864433+0000","last_clean":"2024-09-11T00:22:28.864433+0000","last_became_active":"2024-09-11T00:20:37.441555+0000","last_became_peered":"2024-09-11T00:20:37.441555+0000","last_unstale":"2024-09-11T00:22:28.864433+0000","last_undegraded":"2024-09-11T00:22:28.864433+0000","last_fullsized":"2024-09-11T00:22:28.864433+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:08.155763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863627+0000","last_change":"2024-09-11T00:20:27.071533+0000","last_active":"2024-09-11T00:22:28.863627+0000","last_peered":"2024-09-11T00:22:28.863627+0000","last_clean":"2024-09-11T00:22:28.863627+0000","last_became_active":"2024-09-11T00:20:27.068771+0000","last_became_peered":"2024-09-11T00:20:27.068771+0000","last_unstale":"2024-09-11T00:22:28.863627+0000","last_undegraded":"2024-09-11T00:22:28.863627+0000","last_fullsized":"2024-09-11T00:22:28.863627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:05.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679472+0000","last_change":"2024-09-11T00:20:24.023282+0000","last_active":"2024-09-11T00:22:28.679472+0000","last_peered":"2024-09-11T00:22:28.679472+0000","last_clean":"2024-09-11T00:22:28.679472+0000","last_became_active":"2024-09-11T00:20:24.022914+0000","last_became_peered":"2024-09-11T00:20:24.022914+0000","last_unstale":"2024-09-11T00:22:28.679472+0000","last_undegraded":"2024-09-11T00:22:28.679472+0000","last_fullsized":"2024-09-11T00:22:28.679472+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:45.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864723+0000","last_change":"2024-09-11T00:20:37.457338+0000","last_active":"2024-09-11T00:22:28.864723+0000","last_peered":"2024-09-11T00:22:28.864723+0000","last_clean":"2024-09-11T00:22:28.864723+0000","last_became_active":"2024-09-11T00:20:37.456993+0000","last_became_peered":"2024-09-11T00:20:37.456993+0000","last_unstale":"2024-09-11T00:22:28.864723+0000","last_undegraded":"2024-09-11T00:22:28.864723+0000","last_fullsized":"2024-09-11T00:22:28.864723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:57.284412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864064+0000","last_change":"2024-09-11T00:20:27.062752+0000","last_active":"2024-09-11T00:22:28.864064+0000","last_peered":"2024-09-11T00:22:28.864064+0000","last_clean":"2024-09-11T00:22:28.864064+0000","last_became_active":"2024-09-11T00:20:27.061569+0000","last_became_peered":"2024-09-11T00:20:27.061569+0000","last_unstale":"2024-09-11T00:22:28.864064+0000","last_undegraded":"2024-09-11T00:22:28.864064+0000","last_fullsized":"2024-09-11T00:22:28.864064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:30.904423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864019+0000","last_change":"2024-09-11T00:20:24.023197+0000","last_active":"2024-09-11T00:22:28.864019+0000","last_peered":"2024-09-11T00:22:28.864019+0000","last_clean":"2024-09-11T00:22:28.864019+0000","last_became_active":"2024-09-11T00:20:24.022867+0000","last_became_peered":"2024-09-11T00:20:24.022867+0000","last_unstale":"2024-09-11T00:22:28.864019+0000","last_undegraded":"2024-09-11T00:22:28.864019+0000","last_fullsized":"2024-09-11T00:22:28.864019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:37.949437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680610+0000","last_change":"2024-09-11T00:20:37.451731+0000","last_active":"2024-09-11T00:22:28.680610+0000","last_peered":"2024-09-11T00:22:28.680610+0000","last_clean":"2024-09-11T00:22:28.680610+0000","last_became_active":"2024-09-11T00:20:37.447710+0000","last_became_peered":"2024-09-11T00:20:37.447710+0000","last_unstale":"2024-09-11T00:22:28.680610+0000","last_undegraded":"2024-09-11T00:22:28.680610+0000","last_fullsized":"2024-09-11T00:22:28.680610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:11:38.471216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863635+0000","last_change":"2024-09-11T00:20:27.066021+0000","last_active":"2024-09-11T00:22:28.863635+0000","last_peered":"2024-09-11T00:22:28.863635+0000","last_clean":"2024-09-11T00:22:28.863635+0000","last_became_active":"2024-09-11T00:20:27.065652+0000","last_became_peered":"2024-09-11T00:20:27.065652+0000","last_unstale":"2024-09-11T00:22:28.863635+0000","last_undegraded":"2024-09-11T00:22:28.863635+0000","last_fullsized":"2024-09-11T00:22:28.863635+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:01.933031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864083+0000","last_change":"2024-09-11T00:20:24.035619+0000","last_active":"2024-09-11T00:22:28.864083+0000","last_peered":"2024-09-11T00:22:28.864083+0000","last_clean":"2024-09-11T00:22:28.864083+0000","last_became_active":"2024-09-11T00:20:24.035358+0000","last_became_peered":"2024-09-11T00:20:24.035358+0000","last_unstale":"2024-09-11T00:22:28.864083+0000","last_undegraded":"2024-09-11T00:22:28.864083+0000","last_fullsized":"2024-09-11T00:22:28.864083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:56.890058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864427+0000","last_change":"2024-09-11T00:21:28.226415+0000","last_active":"2024-09-11T00:22:28.864427+0000","last_peered":"2024-09-11T00:22:28.864427+0000","last_clean":"2024-09-11T00:22:28.864427+0000","last_became_active":"2024-09-11T00:21:28.225894+0000","last_became_peered":"2024-09-11T00:21:28.225894+0000","last_unstale":"2024-09-11T00:22:28.864427+0000","last_undegraded":"2024-09-11T00:22:28.864427+0000","last_fullsized":"2024-09-11T00:22:28.864427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:59.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866953+0000","last_change":"2024-09-11T00:22:28.866891+0000","last_active":"2024-09-11T00:22:28.866953+0000","last_peered":"2024-09-11T00:22:28.866953+0000","last_clean":"2024-09-11T00:22:28.866953+0000","last_became_active":"2024-09-11T00:22:28.866541+0000","last_became_peered":"2024-09-11T00:22:28.866541+0000","last_unstale":"2024-09-11T00:22:28.866953+0000","last_undegraded":"2024-09-11T00:22:28.866953+0000","last_fullsized":"2024-09-11T00:22:28.866953+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:35.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865567+0000","last_change":"2024-09-11T00:20:24.535701+0000","last_active":"2024-09-11T00:22:28.865567+0000","last_peered":"2024-09-11T00:22:28.865567+0000","last_clean":"2024-09-11T00:22:28.865567+0000","last_became_active":"2024-09-11T00:20:24.535389+0000","last_became_peered":"2024-09-11T00:20:24.535389+0000","last_unstale":"2024-09-11T00:22:28.865567+0000","last_undegraded":"2024-09-11T00:22:28.865567+0000","last_fullsized":"2024-09-11T00:22:28.865567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:29.044747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679581+0000","last_change":"2024-09-11T00:20:37.435584+0000","last_active":"2024-09-11T00:22:28.679581+0000","last_peered":"2024-09-11T00:22:28.679581+0000","last_clean":"2024-09-11T00:22:28.679581+0000","last_became_active":"2024-09-11T00:20:37.435237+0000","last_became_peered":"2024-09-11T00:20:37.435237+0000","last_unstale":"2024-09-11T00:22:28.679581+0000","last_undegraded":"2024-09-11T00:22:28.679581+0000","last_fullsized":"2024-09-11T00:22:28.679581+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:34.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863962+0000","last_change":"2024-09-11T00:20:27.068718+0000","last_active":"2024-09-11T00:22:28.863962+0000","last_peered":"2024-09-11T00:22:28.863962+0000","last_clean":"2024-09-11T00:22:28.863962+0000","last_became_active":"2024-09-11T00:20:27.058921+0000","last_became_peered":"2024-09-11T00:20:27.058921+0000","last_unstale":"2024-09-11T00:22:28.863962+0000","last_undegraded":"2024-09-11T00:22:28.863962+0000","last_fullsized":"2024-09-11T00:22:28.863962+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:16.056742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'12","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680891+0000","last_change":"2024-09-11T00:20:24.023580+0000","last_active":"2024-09-11T00:22:28.680891+0000","last_peered":"2024-09-11T00:22:28.680891+0000","last_clean":"2024-09-11T00:22:28.680891+0000","last_became_active":"2024-09-11T00:20:24.023029+0000","last_became_peered":"2024-09-11T00:20:24.023029+0000","last_unstale":"2024-09-11T00:22:28.680891+0000","last_undegraded":"2024-09-11T00:22:28.680891+0000","last_fullsized":"2024-09-11T00:22:28.680891+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:05.574228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863409+0000","last_change":"2024-09-11T00:20:37.449229+0000","last_active":"2024-09-11T00:22:28.863409+0000","last_peered":"2024-09-11T00:22:28.863409+0000","last_clean":"2024-09-11T00:22:28.863409+0000","last_became_active":"2024-09-11T00:20:37.447464+0000","last_became_peered":"2024-09-11T00:20:37.447464+0000","last_unstale":"2024-09-11T00:22:28.863409+0000","last_undegraded":"2024-09-11T00:22:28.863409+0000","last_fullsized":"2024-09-11T00:22:28.863409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:47.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683065+0000","last_change":"2024-09-11T00:20:27.063518+0000","last_active":"2024-09-11T00:22:28.683065+0000","last_peered":"2024-09-11T00:22:28.683065+0000","last_clean":"2024-09-11T00:22:28.683065+0000","last_became_active":"2024-09-11T00:20:27.063122+0000","last_became_peered":"2024-09-11T00:20:27.063122+0000","last_unstale":"2024-09-11T00:22:28.683065+0000","last_undegraded":"2024-09-11T00:22:28.683065+0000","last_fullsized":"2024-09-11T00:22:28.683065+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:52.269862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864785+0000","last_change":"2024-09-11T00:20:24.035835+0000","last_active":"2024-09-11T00:22:28.864785+0000","last_peered":"2024-09-11T00:22:28.864785+0000","last_clean":"2024-09-11T00:22:28.864785+0000","last_became_active":"2024-09-11T00:20:24.035471+0000","last_became_peered":"2024-09-11T00:20:24.035471+0000","last_unstale":"2024-09-11T00:22:28.864785+0000","last_undegraded":"2024-09-11T00:22:28.864785+0000","last_fullsized":"2024-09-11T00:22:28.864785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:37.114863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863849+0000","last_change":"2024-09-11T00:20:37.439609+0000","last_active":"2024-09-11T00:22:28.863849+0000","last_peered":"2024-09-11T00:22:28.863849+0000","last_clean":"2024-09-11T00:22:28.863849+0000","last_became_active":"2024-09-11T00:20:37.439139+0000","last_became_peered":"2024-09-11T00:20:37.439139+0000","last_unstale":"2024-09-11T00:22:28.863849+0000","last_undegraded":"2024-09-11T00:22:28.863849+0000","last_fullsized":"2024-09-11T00:22:28.863849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:19.152096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830271+0000","last_change":"2024-09-11T00:20:27.062879+0000","last_active":"2024-09-11T00:22:28.830271+0000","last_peered":"2024-09-11T00:22:28.830271+0000","last_clean":"2024-09-11T00:22:28.830271+0000","last_became_active":"2024-09-11T00:20:27.062200+0000","last_became_peered":"2024-09-11T00:20:27.062200+0000","last_unstale":"2024-09-11T00:22:28.830271+0000","last_undegraded":"2024-09-11T00:22:28.830271+0000","last_fullsized":"2024-09-11T00:22:28.830271+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:57.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680751+0000","last_change":"2024-09-11T00:21:28.230802+0000","last_active":"2024-09-11T00:22:28.680751+0000","last_peered":"2024-09-11T00:22:28.680751+0000","last_clean":"2024-09-11T00:22:28.680751+0000","last_became_active":"2024-09-11T00:21:28.230378+0000","last_became_peered":"2024-09-11T00:21:28.230378+0000","last_unstale":"2024-09-11T00:22:28.680751+0000","last_undegraded":"2024-09-11T00:22:28.680751+0000","last_fullsized":"2024-09-11T00:22:28.680751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:35.676964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864366+0000","last_change":"2024-09-11T00:20:37.444912+0000","last_active":"2024-09-11T00:22:28.864366+0000","last_peered":"2024-09-11T00:22:28.864366+0000","last_clean":"2024-09-11T00:22:28.864366+0000","last_became_active":"2024-09-11T00:20:37.444472+0000","last_became_peered":"2024-09-11T00:20:37.444472+0000","last_unstale":"2024-09-11T00:22:28.864366+0000","last_undegraded":"2024-09-11T00:22:28.864366+0000","last_fullsized":"2024-09-11T00:22:28.864366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:23.263981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830293+0000","last_change":"2024-09-11T00:20:27.060477+0000","last_active":"2024-09-11T00:22:28.830293+0000","last_peered":"2024-09-11T00:22:28.830293+0000","last_clean":"2024-09-11T00:22:28.830293+0000","last_became_active":"2024-09-11T00:20:27.060128+0000","last_became_peered":"2024-09-11T00:20:27.060128+0000","last_unstale":"2024-09-11T00:22:28.830293+0000","last_undegraded":"2024-09-11T00:22:28.830293+0000","last_fullsized":"2024-09-11T00:22:28.830293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:55.967799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866156+0000","last_change":"2024-09-11T00:20:24.028229+0000","last_active":"2024-09-11T00:22:28.866156+0000","last_peered":"2024-09-11T00:22:28.866156+0000","last_clean":"2024-09-11T00:22:28.866156+0000","last_became_active":"2024-09-11T00:20:24.027832+0000","last_became_peered":"2024-09-11T00:20:24.027832+0000","last_unstale":"2024-09-11T00:22:28.866156+0000","last_undegraded":"2024-09-11T00:22:28.866156+0000","last_fullsized":"2024-09-11T00:22:28.866156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:19.960403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682825+0000","last_change":"2024-09-11T00:20:37.442414+0000","last_active":"2024-09-11T00:22:28.682825+0000","last_peered":"2024-09-11T00:22:28.682825+0000","last_clean":"2024-09-11T00:22:28.682825+0000","last_became_active":"2024-09-11T00:20:37.442021+0000","last_became_peered":"2024-09-11T00:20:37.442021+0000","last_unstale":"2024-09-11T00:22:28.682825+0000","last_undegraded":"2024-09-11T00:22:28.682825+0000","last_fullsized":"2024-09-11T00:22:28.682825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:21.296638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680259+0000","last_change":"2024-09-11T00:21:28.233303+0000","last_active":"2024-09-11T00:22:28.680259+0000","last_peered":"2024-09-11T00:22:28.680259+0000","last_clean":"2024-09-11T00:22:28.680259+0000","last_became_active":"2024-09-11T00:21:28.232039+0000","last_became_peered":"2024-09-11T00:21:28.232039+0000","last_unstale":"2024-09-11T00:22:28.680259+0000","last_undegraded":"2024-09-11T00:22:28.680259+0000","last_fullsized":"2024-09-11T00:22:28.680259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:36.828102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:34.321082+0000","last_change":"2024-09-11T00:20:24.037218+0000","last_active":"2024-09-11T00:22:34.321082+0000","last_peered":"2024-09-11T00:22:34.321082+0000","last_clean":"2024-09-11T00:22:34.321082+0000","last_became_active":"2024-09-11T00:20:24.036005+0000","last_became_peered":"2024-09-11T00:20:24.036005+0000","last_unstale":"2024-09-11T00:22:34.321082+0000","last_undegraded":"2024-09-11T00:22:34.321082+0000","last_fullsized":"2024-09-11T00:22:34.321082+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:17.758812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683029+0000","last_change":"2024-09-11T00:20:37.445584+0000","last_active":"2024-09-11T00:22:28.683029+0000","last_peered":"2024-09-11T00:22:28.683029+0000","last_clean":"2024-09-11T00:22:28.683029+0000","last_became_active":"2024-09-11T00:20:37.445196+0000","last_became_peered":"2024-09-11T00:20:37.445196+0000","last_unstale":"2024-09-11T00:22:28.683029+0000","last_undegraded":"2024-09-11T00:22:28.683029+0000","last_fullsized":"2024-09-11T00:22:28.683029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:05.966282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682632+0000","last_change":"2024-09-11T00:20:27.060428+0000","last_active":"2024-09-11T00:22:28.682632+0000","last_peered":"2024-09-11T00:22:28.682632+0000","last_clean":"2024-09-11T00:22:28.682632+0000","last_became_active":"2024-09-11T00:20:27.060057+0000","last_became_peered":"2024-09-11T00:20:27.060057+0000","last_unstale":"2024-09-11T00:22:28.682632+0000","last_undegraded":"2024-09-11T00:22:28.682632+0000","last_fullsized":"2024-09-11T00:22:28.682632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:37.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865091+0000","last_change":"2024-09-11T00:20:24.025496+0000","last_active":"2024-09-11T00:22:28.865091+0000","last_peered":"2024-09-11T00:22:28.865091+0000","last_clean":"2024-09-11T00:22:28.865091+0000","last_became_active":"2024-09-11T00:20:24.025158+0000","last_became_peered":"2024-09-11T00:20:24.025158+0000","last_unstale":"2024-09-11T00:22:28.865091+0000","last_undegraded":"2024-09-11T00:22:28.865091+0000","last_fullsized":"2024-09-11T00:22:28.865091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:40.439708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734388+0000","last_change":"2024-09-11T00:20:37.453118+0000","last_active":"2024-09-11T00:22:28.734388+0000","last_peered":"2024-09-11T00:22:28.734388+0000","last_clean":"2024-09-11T00:22:28.734388+0000","last_became_active":"2024-09-11T00:20:37.452669+0000","last_became_peered":"2024-09-11T00:20:37.452669+0000","last_unstale":"2024-09-11T00:22:28.734388+0000","last_undegraded":"2024-09-11T00:22:28.734388+0000","last_fullsized":"2024-09-11T00:22:28.734388+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:51:58.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863822+0000","last_change":"2024-09-11T00:20:27.069708+0000","last_active":"2024-09-11T00:22:28.863822+0000","last_peered":"2024-09-11T00:22:28.863822+0000","last_clean":"2024-09-11T00:22:28.863822+0000","last_became_active":"2024-09-11T00:20:27.067500+0000","last_became_peered":"2024-09-11T00:20:27.067500+0000","last_unstale":"2024-09-11T00:22:28.863822+0000","last_undegraded":"2024-09-11T00:22:28.863822+0000","last_fullsized":"2024-09-11T00:22:28.863822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:57.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864981+0000","last_change":"2024-09-11T00:20:24.023504+0000","last_active":"2024-09-11T00:22:28.864981+0000","last_peered":"2024-09-11T00:22:28.864981+0000","last_clean":"2024-09-11T00:22:28.864981+0000","last_became_active":"2024-09-11T00:20:24.022706+0000","last_became_peered":"2024-09-11T00:20:24.022706+0000","last_unstale":"2024-09-11T00:22:28.864981+0000","last_undegraded":"2024-09-11T00:22:28.864981+0000","last_fullsized":"2024-09-11T00:22:28.864981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:21.216325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865351+0000","last_change":"2024-09-11T00:19:05.531191+0000","last_active":"2024-09-11T00:22:28.865351+0000","last_peered":"2024-09-11T00:22:28.865351+0000","last_clean":"2024-09-11T00:22:28.865351+0000","last_became_active":"2024-09-11T00:19:05.525664+0000","last_became_peered":"2024-09-11T00:19:05.525664+0000","last_unstale":"2024-09-11T00:22:28.865351+0000","last_undegraded":"2024-09-11T00:22:28.865351+0000","last_fullsized":"2024-09-11T00:22:28.865351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_clean_scrub_stamp":"2024-09-11T00:15:23.365345+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:06.253537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865876+0000","last_change":"2024-09-11T00:20:37.446403+0000","last_active":"2024-09-11T00:22:28.865876+0000","last_peered":"2024-09-11T00:22:28.865876+0000","last_clean":"2024-09-11T00:22:28.865876+0000","last_became_active":"2024-09-11T00:20:37.446029+0000","last_became_peered":"2024-09-11T00:20:37.446029+0000","last_unstale":"2024-09-11T00:22:28.865876+0000","last_undegraded":"2024-09-11T00:22:28.865876+0000","last_fullsized":"2024-09-11T00:22:28.865876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866106+0000","last_change":"2024-09-11T00:20:27.063086+0000","last_active":"2024-09-11T00:22:28.866106+0000","last_peered":"2024-09-11T00:22:28.866106+0000","last_clean":"2024-09-11T00:22:28.866106+0000","last_became_active":"2024-09-11T00:20:27.062790+0000","last_became_peered":"2024-09-11T00:20:27.062790+0000","last_unstale":"2024-09-11T00:22:28.866106+0000","last_undegraded":"2024-09-11T00:22:28.866106+0000","last_fullsized":"2024-09-11T00:22:28.866106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:36.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679590+0000","last_change":"2024-09-11T00:20:24.535837+0000","last_active":"2024-09-11T00:22:28.679590+0000","last_peered":"2024-09-11T00:22:28.679590+0000","last_clean":"2024-09-11T00:22:28.679590+0000","last_became_active":"2024-09-11T00:20:24.535308+0000","last_became_peered":"2024-09-11T00:20:24.535308+0000","last_unstale":"2024-09-11T00:22:28.679590+0000","last_undegraded":"2024-09-11T00:22:28.679590+0000","last_fullsized":"2024-09-11T00:22:28.679590+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.807570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864182+0000","last_change":"2024-09-11T00:20:37.443844+0000","last_active":"2024-09-11T00:22:28.864182+0000","last_peered":"2024-09-11T00:22:28.864182+0000","last_clean":"2024-09-11T00:22:28.864182+0000","last_became_active":"2024-09-11T00:20:37.443374+0000","last_became_peered":"2024-09-11T00:20:37.443374+0000","last_unstale":"2024-09-11T00:22:28.864182+0000","last_undegraded":"2024-09-11T00:22:28.864182+0000","last_fullsized":"2024-09-11T00:22:28.864182+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:47.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865704+0000","last_change":"2024-09-11T00:20:27.072874+0000","last_active":"2024-09-11T00:22:28.865704+0000","last_peered":"2024-09-11T00:22:28.865704+0000","last_clean":"2024-09-11T00:22:28.865704+0000","last_became_active":"2024-09-11T00:20:27.072454+0000","last_became_peered":"2024-09-11T00:20:27.072454+0000","last_unstale":"2024-09-11T00:22:28.865704+0000","last_undegraded":"2024-09-11T00:22:28.865704+0000","last_fullsized":"2024-09-11T00:22:28.865704+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:25.731037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734627+0000","last_change":"2024-09-11T00:20:24.018929+0000","last_active":"2024-09-11T00:22:28.734627+0000","last_peered":"2024-09-11T00:22:28.734627+0000","last_clean":"2024-09-11T00:22:28.734627+0000","last_became_active":"2024-09-11T00:20:24.018575+0000","last_became_peered":"2024-09-11T00:20:24.018575+0000","last_unstale":"2024-09-11T00:22:28.734627+0000","last_undegraded":"2024-09-11T00:22:28.734627+0000","last_fullsized":"2024-09-11T00:22:28.734627+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.695561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679036+0000","last_change":"2024-09-11T00:20:37.453468+0000","last_active":"2024-09-11T00:22:28.679036+0000","last_peered":"2024-09-11T00:22:28.679036+0000","last_clean":"2024-09-11T00:22:28.679036+0000","last_became_active":"2024-09-11T00:20:37.453007+0000","last_became_peered":"2024-09-11T00:20:37.453007+0000","last_unstale":"2024-09-11T00:22:28.679036+0000","last_undegraded":"2024-09-11T00:22:28.679036+0000","last_fullsized":"2024-09-11T00:22:28.679036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:22.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865897+0000","last_change":"2024-09-11T00:20:27.064618+0000","last_active":"2024-09-11T00:22:28.865897+0000","last_peered":"2024-09-11T00:22:28.865897+0000","last_clean":"2024-09-11T00:22:28.865897+0000","last_became_active":"2024-09-11T00:20:27.064310+0000","last_became_peered":"2024-09-11T00:20:27.064310+0000","last_unstale":"2024-09-11T00:22:28.865897+0000","last_undegraded":"2024-09-11T00:22:28.865897+0000","last_fullsized":"2024-09-11T00:22:28.865897+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:41.320561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682985+0000","last_change":"2024-09-11T00:20:24.023522+0000","last_active":"2024-09-11T00:22:28.682985+0000","last_peered":"2024-09-11T00:22:28.682985+0000","last_clean":"2024-09-11T00:22:28.682985+0000","last_became_active":"2024-09-11T00:20:24.022694+0000","last_became_peered":"2024-09-11T00:20:24.022694+0000","last_unstale":"2024-09-11T00:22:28.682985+0000","last_undegraded":"2024-09-11T00:22:28.682985+0000","last_fullsized":"2024-09-11T00:22:28.682985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:12.374179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733868+0000","last_change":"2024-09-11T00:20:37.441239+0000","last_active":"2024-09-11T00:22:28.733868+0000","last_peered":"2024-09-11T00:22:28.733868+0000","last_clean":"2024-09-11T00:22:28.733868+0000","last_became_active":"2024-09-11T00:20:37.440825+0000","last_became_peered":"2024-09-11T00:20:37.440825+0000","last_unstale":"2024-09-11T00:22:28.733868+0000","last_undegraded":"2024-09-11T00:22:28.733868+0000","last_fullsized":"2024-09-11T00:22:28.733868+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:04.972569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864299+0000","last_change":"2024-09-11T00:20:27.059780+0000","last_active":"2024-09-11T00:22:28.864299+0000","last_peered":"2024-09-11T00:22:28.864299+0000","last_clean":"2024-09-11T00:22:28.864299+0000","last_became_active":"2024-09-11T00:20:27.059411+0000","last_became_peered":"2024-09-11T00:20:27.059411+0000","last_unstale":"2024-09-11T00:22:28.864299+0000","last_undegraded":"2024-09-11T00:22:28.864299+0000","last_fullsized":"2024-09-11T00:22:28.864299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:42.058883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864457+0000","last_change":"2024-09-11T00:20:24.024115+0000","last_active":"2024-09-11T00:22:28.864457+0000","last_peered":"2024-09-11T00:22:28.864457+0000","last_clean":"2024-09-11T00:22:28.864457+0000","last_became_active":"2024-09-11T00:20:24.023729+0000","last_became_peered":"2024-09-11T00:20:24.023729+0000","last_unstale":"2024-09-11T00:22:28.864457+0000","last_undegraded":"2024-09-11T00:22:28.864457+0000","last_fullsized":"2024-09-11T00:22:28.864457+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:15.874168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733847+0000","last_change":"2024-09-11T00:20:37.443591+0000","last_active":"2024-09-11T00:22:28.733847+0000","last_peered":"2024-09-11T00:22:28.733847+0000","last_clean":"2024-09-11T00:22:28.733847+0000","last_became_active":"2024-09-11T00:20:37.443115+0000","last_became_peered":"2024-09-11T00:20:37.443115+0000","last_unstale":"2024-09-11T00:22:28.733847+0000","last_undegraded":"2024-09-11T00:22:28.733847+0000","last_fullsized":"2024-09-11T00:22:28.733847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:34.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734026+0000","last_change":"2024-09-11T00:20:27.061676+0000","last_active":"2024-09-11T00:22:28.734026+0000","last_peered":"2024-09-11T00:22:28.734026+0000","last_clean":"2024-09-11T00:22:28.734026+0000","last_became_active":"2024-09-11T00:20:27.061372+0000","last_became_peered":"2024-09-11T00:20:27.061372+0000","last_unstale":"2024-09-11T00:22:28.734026+0000","last_undegraded":"2024-09-11T00:22:28.734026+0000","last_fullsized":"2024-09-11T00:22:28.734026+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:24.296673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863776+0000","last_change":"2024-09-11T00:21:28.244227+0000","last_active":"2024-09-11T00:22:28.863776+0000","last_peered":"2024-09-11T00:22:28.863776+0000","last_clean":"2024-09-11T00:22:28.863776+0000","last_became_active":"2024-09-11T00:21:28.231547+0000","last_became_peered":"2024-09-11T00:21:28.231547+0000","last_unstale":"2024-09-11T00:22:28.863776+0000","last_undegraded":"2024-09-11T00:22:28.863776+0000","last_fullsized":"2024-09-11T00:22:28.863776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:35.119308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682940+0000","last_change":"2024-09-11T00:20:37.454603+0000","last_active":"2024-09-11T00:22:28.682940+0000","last_peered":"2024-09-11T00:22:28.682940+0000","last_clean":"2024-09-11T00:22:28.682940+0000","last_became_active":"2024-09-11T00:20:37.454123+0000","last_became_peered":"2024-09-11T00:20:37.454123+0000","last_unstale":"2024-09-11T00:22:28.682940+0000","last_undegraded":"2024-09-11T00:22:28.682940+0000","last_fullsized":"2024-09-11T00:22:28.682940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:09.111768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864882+0000","last_change":"2024-09-11T00:20:27.058806+0000","last_active":"2024-09-11T00:22:28.864882+0000","last_peered":"2024-09-11T00:22:28.864882+0000","last_clean":"2024-09-11T00:22:28.864882+0000","last_became_active":"2024-09-11T00:20:27.058458+0000","last_became_peered":"2024-09-11T00:20:27.058458+0000","last_unstale":"2024-09-11T00:22:28.864882+0000","last_undegraded":"2024-09-11T00:22:28.864882+0000","last_fullsized":"2024-09-11T00:22:28.864882+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:48.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:32.318842+0000","last_change":"2024-09-11T00:20:24.022328+0000","last_active":"2024-09-11T00:22:32.318842+0000","last_peered":"2024-09-11T00:22:32.318842+0000","last_clean":"2024-09-11T00:22:32.318842+0000","last_became_active":"2024-09-11T00:20:24.021908+0000","last_became_peered":"2024-09-11T00:20:24.021908+0000","last_unstale":"2024-09-11T00:22:32.318842+0000","last_undegraded":"2024-09-11T00:22:32.318842+0000","last_fullsized":"2024-09-11T00:22:32.318842+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:35.688767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680728+0000","last_change":"2024-09-11T00:20:37.444898+0000","last_active":"2024-09-11T00:22:28.680728+0000","last_peered":"2024-09-11T00:22:28.680728+0000","last_clean":"2024-09-11T00:22:28.680728+0000","last_became_active":"2024-09-11T00:20:37.443840+0000","last_became_peered":"2024-09-11T00:20:37.443840+0000","last_unstale":"2024-09-11T00:22:28.680728+0000","last_undegraded":"2024-09-11T00:22:28.680728+0000","last_fullsized":"2024-09-11T00:22:28.680728+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:58.668882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865608+0000","last_change":"2024-09-11T00:20:27.061198+0000","last_active":"2024-09-11T00:22:28.865608+0000","last_peered":"2024-09-11T00:22:28.865608+0000","last_clean":"2024-09-11T00:22:28.865608+0000","last_became_active":"2024-09-11T00:20:27.060877+0000","last_became_peered":"2024-09-11T00:20:27.060877+0000","last_unstale":"2024-09-11T00:22:28.865608+0000","last_undegraded":"2024-09-11T00:22:28.865608+0000","last_fullsized":"2024-09-11T00:22:28.865608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:03.595262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734204+0000","last_change":"2024-09-11T00:20:24.022755+0000","last_active":"2024-09-11T00:22:28.734204+0000","last_peered":"2024-09-11T00:22:28.734204+0000","last_clean":"2024-09-11T00:22:28.734204+0000","last_became_active":"2024-09-11T00:20:24.021530+0000","last_became_peered":"2024-09-11T00:20:24.021530+0000","last_unstale":"2024-09-11T00:22:28.734204+0000","last_undegraded":"2024-09-11T00:22:28.734204+0000","last_fullsized":"2024-09-11T00:22:28.734204+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:33.881000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864289+0000","last_change":"2024-09-11T00:20:37.443722+0000","last_active":"2024-09-11T00:22:28.864289+0000","last_peered":"2024-09-11T00:22:28.864289+0000","last_clean":"2024-09-11T00:22:28.864289+0000","last_became_active":"2024-09-11T00:20:37.443339+0000","last_became_peered":"2024-09-11T00:20:37.443339+0000","last_unstale":"2024-09-11T00:22:28.864289+0000","last_undegraded":"2024-09-11T00:22:28.864289+0000","last_fullsized":"2024-09-11T00:22:28.864289+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:30.221804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682691+0000","last_change":"2024-09-11T00:20:27.073158+0000","last_active":"2024-09-11T00:22:28.682691+0000","last_peered":"2024-09-11T00:22:28.682691+0000","last_clean":"2024-09-11T00:22:28.682691+0000","last_became_active":"2024-09-11T00:20:27.072805+0000","last_became_peered":"2024-09-11T00:20:27.072805+0000","last_unstale":"2024-09-11T00:22:28.682691+0000","last_undegraded":"2024-09-11T00:22:28.682691+0000","last_fullsized":"2024-09-11T00:22:28.682691+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:35.124632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679649+0000","last_change":"2024-09-11T00:20:24.027178+0000","last_active":"2024-09-11T00:22:28.679649+0000","last_peered":"2024-09-11T00:22:28.679649+0000","last_clean":"2024-09-11T00:22:28.679649+0000","last_became_active":"2024-09-11T00:20:24.026821+0000","last_became_peered":"2024-09-11T00:20:24.026821+0000","last_unstale":"2024-09-11T00:22:28.679649+0000","last_undegraded":"2024-09-11T00:22:28.679649+0000","last_fullsized":"2024-09-11T00:22:28.679649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:38.393135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734368+0000","last_change":"2024-09-11T00:20:37.447587+0000","last_active":"2024-09-11T00:22:28.734368+0000","last_peered":"2024-09-11T00:22:28.734368+0000","last_clean":"2024-09-11T00:22:28.734368+0000","last_became_active":"2024-09-11T00:20:37.447166+0000","last_became_peered":"2024-09-11T00:20:37.447166+0000","last_unstale":"2024-09-11T00:22:28.734368+0000","last_undegraded":"2024-09-11T00:22:28.734368+0000","last_fullsized":"2024-09-11T00:22:28.734368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:47.469539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864619+0000","last_change":"2024-09-11T00:20:27.059914+0000","last_active":"2024-09-11T00:22:28.864619+0000","last_peered":"2024-09-11T00:22:28.864619+0000","last_clean":"2024-09-11T00:22:28.864619+0000","last_became_active":"2024-09-11T00:20:27.059595+0000","last_became_peered":"2024-09-11T00:20:27.059595+0000","last_unstale":"2024-09-11T00:22:28.864619+0000","last_undegraded":"2024-09-11T00:22:28.864619+0000","last_fullsized":"2024-09-11T00:22:28.864619+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.360010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":225,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:33.333900+0000","last_change":"2024-09-11T00:20:24.023094+0000","last_active":"2024-09-11T00:22:33.333900+0000","last_peered":"2024-09-11T00:22:33.333900+0000","last_clean":"2024-09-11T00:22:33.333900+0000","last_became_active":"2024-09-11T00:20:24.022742+0000","last_became_peered":"2024-09-11T00:20:24.022742+0000","last_unstale":"2024-09-11T00:22:33.333900+0000","last_undegraded":"2024-09-11T00:22:33.333900+0000","last_fullsized":"2024-09-11T00:22:33.333900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:41.354435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":147,"num_read_kb":134,"num_write":34,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680544+0000","last_change":"2024-09-11T00:20:37.451610+0000","last_active":"2024-09-11T00:22:28.680544+0000","last_peered":"2024-09-11T00:22:28.680544+0000","last_clean":"2024-09-11T00:22:28.680544+0000","last_became_active":"2024-09-11T00:20:37.444233+0000","last_became_peered":"2024-09-11T00:20:37.444233+0000","last_unstale":"2024-09-11T00:22:28.680544+0000","last_undegraded":"2024-09-11T00:22:28.680544+0000","last_fullsized":"2024-09-11T00:22:28.680544+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:02.558457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864954+0000","last_change":"2024-09-11T00:20:27.061229+0000","last_active":"2024-09-11T00:22:28.864954+0000","last_peered":"2024-09-11T00:22:28.864954+0000","last_clean":"2024-09-11T00:22:28.864954+0000","last_became_active":"2024-09-11T00:20:27.060360+0000","last_became_peered":"2024-09-11T00:20:27.060360+0000","last_unstale":"2024-09-11T00:22:28.864954+0000","last_undegraded":"2024-09-11T00:22:28.864954+0000","last_fullsized":"2024-09-11T00:22:28.864954+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.107405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734515+0000","last_change":"2024-09-11T00:20:24.036483+0000","last_active":"2024-09-11T00:22:28.734515+0000","last_peered":"2024-09-11T00:22:28.734515+0000","last_clean":"2024-09-11T00:22:28.734515+0000","last_became_active":"2024-09-11T00:20:24.035855+0000","last_became_peered":"2024-09-11T00:20:24.035855+0000","last_unstale":"2024-09-11T00:22:28.734515+0000","last_undegraded":"2024-09-11T00:22:28.734515+0000","last_fullsized":"2024-09-11T00:22:28.734515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.611169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864426+0000","last_change":"2024-09-11T00:21:28.245665+0000","last_active":"2024-09-11T00:22:28.864426+0000","last_peered":"2024-09-11T00:22:28.864426+0000","last_clean":"2024-09-11T00:22:28.864426+0000","last_became_active":"2024-09-11T00:21:28.245221+0000","last_became_peered":"2024-09-11T00:21:28.245221+0000","last_unstale":"2024-09-11T00:22:28.864426+0000","last_undegraded":"2024-09-11T00:22:28.864426+0000","last_fullsized":"2024-09-11T00:22:28.864426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:36.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679687+0000","last_change":"2024-09-11T00:21:28.231435+0000","last_active":"2024-09-11T00:22:28.679687+0000","last_peered":"2024-09-11T00:22:28.679687+0000","last_clean":"2024-09-11T00:22:28.679687+0000","last_became_active":"2024-09-11T00:21:28.230956+0000","last_became_peered":"2024-09-11T00:21:28.230956+0000","last_unstale":"2024-09-11T00:22:28.679687+0000","last_undegraded":"2024-09-11T00:22:28.679687+0000","last_fullsized":"2024-09-11T00:22:28.679687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:28.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865489+0000","last_change":"2024-09-11T00:20:24.023068+0000","last_active":"2024-09-11T00:22:28.865489+0000","last_peered":"2024-09-11T00:22:28.865489+0000","last_clean":"2024-09-11T00:22:28.865489+0000","last_became_active":"2024-09-11T00:20:24.022718+0000","last_became_peered":"2024-09-11T00:20:24.022718+0000","last_unstale":"2024-09-11T00:22:28.865489+0000","last_undegraded":"2024-09-11T00:22:28.865489+0000","last_fullsized":"2024-09-11T00:22:28.865489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.651536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863633+0000","last_change":"2024-09-11T00:21:28.231887+0000","last_active":"2024-09-11T00:22:28.863633+0000","last_peered":"2024-09-11T00:22:28.863633+0000","last_clean":"2024-09-11T00:22:28.863633+0000","last_became_active":"2024-09-11T00:21:28.231456+0000","last_became_peered":"2024-09-11T00:21:28.231456+0000","last_unstale":"2024-09-11T00:22:28.863633+0000","last_undegraded":"2024-09-11T00:22:28.863633+0000","last_fullsized":"2024-09-11T00:22:28.863633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:32.712598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865591+0000","last_change":"2024-09-11T00:20:27.065109+0000","last_active":"2024-09-11T00:22:28.865591+0000","last_peered":"2024-09-11T00:22:28.865591+0000","last_clean":"2024-09-11T00:22:28.865591+0000","last_became_active":"2024-09-11T00:20:27.064677+0000","last_became_peered":"2024-09-11T00:20:27.064677+0000","last_unstale":"2024-09-11T00:22:28.865591+0000","last_undegraded":"2024-09-11T00:22:28.865591+0000","last_fullsized":"2024-09-11T00:22:28.865591+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:19.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":117,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:32.318753+0000","last_change":"2024-09-11T00:20:24.025347+0000","last_active":"2024-09-11T00:22:32.318753+0000","last_peered":"2024-09-11T00:22:32.318753+0000","last_clean":"2024-09-11T00:22:32.318753+0000","last_became_active":"2024-09-11T00:20:24.024980+0000","last_became_peered":"2024-09-11T00:20:24.024980+0000","last_unstale":"2024-09-11T00:22:32.318753+0000","last_undegraded":"2024-09-11T00:22:32.318753+0000","last_fullsized":"2024-09-11T00:22:32.318753+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:12.530339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863535+0000","last_change":"2024-09-11T00:20:37.440340+0000","last_active":"2024-09-11T00:22:28.863535+0000","last_peered":"2024-09-11T00:22:28.863535+0000","last_clean":"2024-09-11T00:22:28.863535+0000","last_became_active":"2024-09-11T00:20:37.439896+0000","last_became_peered":"2024-09-11T00:20:37.439896+0000","last_unstale":"2024-09-11T00:22:28.863535+0000","last_undegraded":"2024-09-11T00:22:28.863535+0000","last_fullsized":"2024-09-11T00:22:28.863535+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:38.774065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679146+0000","last_change":"2024-09-11T00:20:37.445860+0000","last_active":"2024-09-11T00:22:28.679146+0000","last_peered":"2024-09-11T00:22:28.679146+0000","last_clean":"2024-09-11T00:22:28.679146+0000","last_became_active":"2024-09-11T00:20:37.445450+0000","last_became_peered":"2024-09-11T00:20:37.445450+0000","last_unstale":"2024-09-11T00:22:28.679146+0000","last_undegraded":"2024-09-11T00:22:28.679146+0000","last_fullsized":"2024-09-11T00:22:28.679146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:44.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680379+0000","last_change":"2024-09-11T00:20:27.073529+0000","last_active":"2024-09-11T00:22:28.680379+0000","last_peered":"2024-09-11T00:22:28.680379+0000","last_clean":"2024-09-11T00:22:28.680379+0000","last_became_active":"2024-09-11T00:20:27.073168+0000","last_became_peered":"2024-09-11T00:20:27.073168+0000","last_unstale":"2024-09-11T00:22:28.680379+0000","last_undegraded":"2024-09-11T00:22:28.680379+0000","last_fullsized":"2024-09-11T00:22:28.680379+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:16.407543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734057+0000","last_change":"2024-09-11T00:20:24.021778+0000","last_active":"2024-09-11T00:22:28.734057+0000","last_peered":"2024-09-11T00:22:28.734057+0000","last_clean":"2024-09-11T00:22:28.734057+0000","last_became_active":"2024-09-11T00:20:24.021443+0000","last_became_peered":"2024-09-11T00:20:24.021443+0000","last_unstale":"2024-09-11T00:22:28.734057+0000","last_undegraded":"2024-09-11T00:22:28.734057+0000","last_fullsized":"2024-09-11T00:22:28.734057+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864345+0000","last_change":"2024-09-11T00:20:37.444279+0000","last_active":"2024-09-11T00:22:28.864345+0000","last_peered":"2024-09-11T00:22:28.864345+0000","last_clean":"2024-09-11T00:22:28.864345+0000","last_became_active":"2024-09-11T00:20:37.443821+0000","last_became_peered":"2024-09-11T00:20:37.443821+0000","last_unstale":"2024-09-11T00:22:28.864345+0000","last_undegraded":"2024-09-11T00:22:28.864345+0000","last_fullsized":"2024-09-11T00:22:28.864345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:10.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863544+0000","last_change":"2024-09-11T00:20:27.066155+0000","last_active":"2024-09-11T00:22:28.863544+0000","last_peered":"2024-09-11T00:22:28.863544+0000","last_clean":"2024-09-11T00:22:28.863544+0000","last_became_active":"2024-09-11T00:20:27.057262+0000","last_became_peered":"2024-09-11T00:20:27.057262+0000","last_unstale":"2024-09-11T00:22:28.863544+0000","last_undegraded":"2024-09-11T00:22:28.863544+0000","last_fullsized":"2024-09-11T00:22:28.863544+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:54.160215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682589+0000","last_change":"2024-09-11T00:20:24.026123+0000","last_active":"2024-09-11T00:22:28.682589+0000","last_peered":"2024-09-11T00:22:28.682589+0000","last_clean":"2024-09-11T00:22:28.682589+0000","last_became_active":"2024-09-11T00:20:24.025832+0000","last_became_peered":"2024-09-11T00:20:24.025832+0000","last_unstale":"2024-09-11T00:22:28.682589+0000","last_undegraded":"2024-09-11T00:22:28.682589+0000","last_fullsized":"2024-09-11T00:22:28.682589+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:45.444840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830114+0000","last_change":"2024-09-11T00:20:24.028523+0000","last_active":"2024-09-11T00:22:28.830114+0000","last_peered":"2024-09-11T00:22:28.830114+0000","last_clean":"2024-09-11T00:22:28.830114+0000","last_became_active":"2024-09-11T00:20:24.027801+0000","last_became_peered":"2024-09-11T00:20:24.027801+0000","last_unstale":"2024-09-11T00:22:28.830114+0000","last_undegraded":"2024-09-11T00:22:28.830114+0000","last_fullsized":"2024-09-11T00:22:28.830114+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:14.174983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734264+0000","last_change":"2024-09-11T00:20:27.074634+0000","last_active":"2024-09-11T00:22:28.734264+0000","last_peered":"2024-09-11T00:22:28.734264+0000","last_clean":"2024-09-11T00:22:28.734264+0000","last_became_active":"2024-09-11T00:20:27.074292+0000","last_became_peered":"2024-09-11T00:20:27.074292+0000","last_unstale":"2024-09-11T00:22:28.734264+0000","last_undegraded":"2024-09-11T00:22:28.734264+0000","last_fullsized":"2024-09-11T00:22:28.734264+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:07.467262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679366+0000","last_change":"2024-09-11T00:20:37.454341+0000","last_active":"2024-09-11T00:22:28.679366+0000","last_peered":"2024-09-11T00:22:28.679366+0000","last_clean":"2024-09-11T00:22:28.679366+0000","last_became_active":"2024-09-11T00:20:37.453868+0000","last_became_peered":"2024-09-11T00:20:37.453868+0000","last_unstale":"2024-09-11T00:22:28.679366+0000","last_undegraded":"2024-09-11T00:22:28.679366+0000","last_fullsized":"2024-09-11T00:22:28.679366+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.015338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":123,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:32.319508+0000","last_change":"2024-09-11T00:20:24.025184+0000","last_active":"2024-09-11T00:22:32.319508+0000","last_peered":"2024-09-11T00:22:32.319508+0000","last_clean":"2024-09-11T00:22:32.319508+0000","last_became_active":"2024-09-11T00:20:24.024670+0000","last_became_peered":"2024-09-11T00:20:24.024670+0000","last_unstale":"2024-09-11T00:22:32.319508+0000","last_undegraded":"2024-09-11T00:22:32.319508+0000","last_fullsized":"2024-09-11T00:22:32.319508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:40.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864184+0000","last_change":"2024-09-11T00:20:27.057822+0000","last_active":"2024-09-11T00:22:28.864184+0000","last_peered":"2024-09-11T00:22:28.864184+0000","last_clean":"2024-09-11T00:22:28.864184+0000","last_became_active":"2024-09-11T00:20:27.057518+0000","last_became_peered":"2024-09-11T00:20:27.057518+0000","last_unstale":"2024-09-11T00:22:28.864184+0000","last_undegraded":"2024-09-11T00:22:28.864184+0000","last_fullsized":"2024-09-11T00:22:28.864184+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:00.264327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679470+0000","last_change":"2024-09-11T00:20:37.438369+0000","last_active":"2024-09-11T00:22:28.679470+0000","last_peered":"2024-09-11T00:22:28.679470+0000","last_clean":"2024-09-11T00:22:28.679470+0000","last_became_active":"2024-09-11T00:20:37.437942+0000","last_became_peered":"2024-09-11T00:20:37.437942+0000","last_unstale":"2024-09-11T00:22:28.679470+0000","last_undegraded":"2024-09-11T00:22:28.679470+0000","last_fullsized":"2024-09-11T00:22:28.679470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.410905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864229+0000","last_change":"2024-09-11T00:20:27.057449+0000","last_active":"2024-09-11T00:22:28.864229+0000","last_peered":"2024-09-11T00:22:28.864229+0000","last_clean":"2024-09-11T00:22:28.864229+0000","last_became_active":"2024-09-11T00:20:27.057146+0000","last_became_peered":"2024-09-11T00:20:27.057146+0000","last_unstale":"2024-09-11T00:22:28.864229+0000","last_undegraded":"2024-09-11T00:22:28.864229+0000","last_fullsized":"2024-09-11T00:22:28.864229+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:57.895062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679881+0000","last_change":"2024-09-11T00:20:27.061515+0000","last_active":"2024-09-11T00:22:28.679881+0000","last_peered":"2024-09-11T00:22:28.679881+0000","last_clean":"2024-09-11T00:22:28.679881+0000","last_became_active":"2024-09-11T00:20:27.061175+0000","last_became_peered":"2024-09-11T00:20:27.061175+0000","last_unstale":"2024-09-11T00:22:28.679881+0000","last_undegraded":"2024-09-11T00:22:28.679881+0000","last_fullsized":"2024-09-11T00:22:28.679881+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:03.112574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679377+0000","last_change":"2024-09-11T00:20:27.063015+0000","last_active":"2024-09-11T00:22:28.679377+0000","last_peered":"2024-09-11T00:22:28.679377+0000","last_clean":"2024-09-11T00:22:28.679377+0000","last_became_active":"2024-09-11T00:20:27.062689+0000","last_became_peered":"2024-09-11T00:20:27.062689+0000","last_unstale":"2024-09-11T00:22:28.679377+0000","last_undegraded":"2024-09-11T00:22:28.679377+0000","last_fullsized":"2024-09-11T00:22:28.679377+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.736073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679199+0000","last_change":"2024-09-11T00:20:27.062196+0000","last_active":"2024-09-11T00:22:28.679199+0000","last_peered":"2024-09-11T00:22:28.679199+0000","last_clean":"2024-09-11T00:22:28.679199+0000","last_became_active":"2024-09-11T00:20:27.061918+0000","last_became_peered":"2024-09-11T00:20:27.061918+0000","last_unstale":"2024-09-11T00:22:28.679199+0000","last_undegraded":"2024-09-11T00:22:28.679199+0000","last_fullsized":"2024-09-11T00:22:28.679199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:45.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682933+0000","last_change":"2024-09-11T00:20:27.059021+0000","last_active":"2024-09-11T00:22:28.682933+0000","last_peered":"2024-09-11T00:22:28.682933+0000","last_clean":"2024-09-11T00:22:28.682933+0000","last_became_active":"2024-09-11T00:20:27.058726+0000","last_became_peered":"2024-09-11T00:20:27.058726+0000","last_unstale":"2024-09-11T00:22:28.682933+0000","last_undegraded":"2024-09-11T00:22:28.682933+0000","last_fullsized":"2024-09-11T00:22:28.682933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:23.900236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864216+0000","last_change":"2024-09-11T00:20:27.057593+0000","last_active":"2024-09-11T00:22:28.864216+0000","last_peered":"2024-09-11T00:22:28.864216+0000","last_clean":"2024-09-11T00:22:28.864216+0000","last_became_active":"2024-09-11T00:20:27.057310+0000","last_became_peered":"2024-09-11T00:20:27.057310+0000","last_unstale":"2024-09-11T00:22:28.864216+0000","last_undegraded":"2024-09-11T00:22:28.864216+0000","last_fullsized":"2024-09-11T00:22:28.864216+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:52.355218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679785+0000","last_change":"2024-09-11T00:20:27.061140+0000","last_active":"2024-09-11T00:22:28.679785+0000","last_peered":"2024-09-11T00:22:28.679785+0000","last_clean":"2024-09-11T00:22:28.679785+0000","last_became_active":"2024-09-11T00:20:27.060500+0000","last_became_peered":"2024-09-11T00:20:27.060500+0000","last_unstale":"2024-09-11T00:22:28.679785+0000","last_undegraded":"2024-09-11T00:22:28.679785+0000","last_fullsized":"2024-09-11T00:22:28.679785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.159068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863940+0000","last_change":"2024-09-11T00:20:27.061936+0000","last_active":"2024-09-11T00:22:28.863940+0000","last_peered":"2024-09-11T00:22:28.863940+0000","last_clean":"2024-09-11T00:22:28.863940+0000","last_became_active":"2024-09-11T00:20:27.061616+0000","last_became_peered":"2024-09-11T00:20:27.061616+0000","last_unstale":"2024-09-11T00:22:28.863940+0000","last_undegraded":"2024-09-11T00:22:28.863940+0000","last_fullsized":"2024-09-11T00:22:28.863940+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:32.185634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733947+0000","last_change":"2024-09-11T00:20:27.074645+0000","last_active":"2024-09-11T00:22:28.733947+0000","last_peered":"2024-09-11T00:22:28.733947+0000","last_clean":"2024-09-11T00:22:28.733947+0000","last_became_active":"2024-09-11T00:20:27.074355+0000","last_became_peered":"2024-09-11T00:20:27.074355+0000","last_unstale":"2024-09-11T00:22:28.733947+0000","last_undegraded":"2024-09-11T00:22:28.733947+0000","last_fullsized":"2024-09-11T00:22:28.733947+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863600+0000","last_change":"2024-09-11T00:20:27.069588+0000","last_active":"2024-09-11T00:22:28.863600+0000","last_peered":"2024-09-11T00:22:28.863600+0000","last_clean":"2024-09-11T00:22:28.863600+0000","last_became_active":"2024-09-11T00:20:27.067403+0000","last_became_peered":"2024-09-11T00:20:27.067403+0000","last_unstale":"2024-09-11T00:22:28.863600+0000","last_undegraded":"2024-09-11T00:22:28.863600+0000","last_fullsized":"2024-09-11T00:22:28.863600+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:29.439013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865193+0000","last_change":"2024-09-11T00:20:27.071586+0000","last_active":"2024-09-11T00:22:28.865193+0000","last_peered":"2024-09-11T00:22:28.865193+0000","last_clean":"2024-09-11T00:22:28.865193+0000","last_became_active":"2024-09-11T00:20:27.071072+0000","last_became_peered":"2024-09-11T00:20:27.071072+0000","last_unstale":"2024-09-11T00:22:28.865193+0000","last_undegraded":"2024-09-11T00:22:28.865193+0000","last_fullsized":"2024-09-11T00:22:28.865193+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:34.133027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683353+0000","last_change":"2024-09-11T00:20:27.066683+0000","last_active":"2024-09-11T00:22:28.683353+0000","last_peered":"2024-09-11T00:22:28.683353+0000","last_clean":"2024-09-11T00:22:28.683353+0000","last_became_active":"2024-09-11T00:20:27.066355+0000","last_became_peered":"2024-09-11T00:20:27.066355+0000","last_unstale":"2024-09-11T00:22:28.683353+0000","last_undegraded":"2024-09-11T00:22:28.683353+0000","last_fullsized":"2024-09-11T00:22:28.683353+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:26.868072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864043+0000","last_change":"2024-09-11T00:20:27.060546+0000","last_active":"2024-09-11T00:22:28.864043+0000","last_peered":"2024-09-11T00:22:28.864043+0000","last_clean":"2024-09-11T00:22:28.864043+0000","last_became_active":"2024-09-11T00:20:27.059619+0000","last_became_peered":"2024-09-11T00:20:27.059619+0000","last_unstale":"2024-09-11T00:22:28.864043+0000","last_undegraded":"2024-09-11T00:22:28.864043+0000","last_fullsized":"2024-09-11T00:22:28.864043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.319854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679643+0000","last_change":"2024-09-11T00:20:27.073342+0000","last_active":"2024-09-11T00:22:28.679643+0000","last_peered":"2024-09-11T00:22:28.679643+0000","last_clean":"2024-09-11T00:22:28.679643+0000","last_became_active":"2024-09-11T00:20:27.072435+0000","last_became_peered":"2024-09-11T00:20:27.072435+0000","last_unstale":"2024-09-11T00:22:28.679643+0000","last_undegraded":"2024-09-11T00:22:28.679643+0000","last_fullsized":"2024-09-11T00:22:28.679643+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:15.943515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680856+0000","last_change":"2024-09-11T00:20:27.063220+0000","last_active":"2024-09-11T00:22:28.680856+0000","last_peered":"2024-09-11T00:22:28.680856+0000","last_clean":"2024-09-11T00:22:28.680856+0000","last_became_active":"2024-09-11T00:20:27.062823+0000","last_became_peered":"2024-09-11T00:20:27.062823+0000","last_unstale":"2024-09-11T00:22:28.680856+0000","last_undegraded":"2024-09-11T00:22:28.680856+0000","last_fullsized":"2024-09-11T00:22:28.680856+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:22.430295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864501+0000","last_change":"2024-09-11T00:20:27.062088+0000","last_active":"2024-09-11T00:22:28.864501+0000","last_peered":"2024-09-11T00:22:28.864501+0000","last_clean":"2024-09-11T00:22:28.864501+0000","last_became_active":"2024-09-11T00:20:27.061639+0000","last_became_peered":"2024-09-11T00:20:27.061639+0000","last_unstale":"2024-09-11T00:22:28.864501+0000","last_undegraded":"2024-09-11T00:22:28.864501+0000","last_fullsized":"2024-09-11T00:22:28.864501+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.965908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863958+0000","last_change":"2024-09-11T00:20:27.067170+0000","last_active":"2024-09-11T00:22:28.863958+0000","last_peered":"2024-09-11T00:22:28.863958+0000","last_clean":"2024-09-11T00:22:28.863958+0000","last_became_active":"2024-09-11T00:20:27.066851+0000","last_became_peered":"2024-09-11T00:20:27.066851+0000","last_unstale":"2024-09-11T00:22:28.863958+0000","last_undegraded":"2024-09-11T00:22:28.863958+0000","last_fullsized":"2024-09-11T00:22:28.863958+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:51.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.678866+0000","last_change":"2024-09-11T00:21:28.233390+0000","last_active":"2024-09-11T00:22:28.678866+0000","last_peered":"2024-09-11T00:22:28.678866+0000","last_clean":"2024-09-11T00:22:28.678866+0000","last_became_active":"2024-09-11T00:21:28.232886+0000","last_became_peered":"2024-09-11T00:21:28.232886+0000","last_unstale":"2024-09-11T00:22:28.678866+0000","last_undegraded":"2024-09-11T00:22:28.678866+0000","last_fullsized":"2024-09-11T00:22:28.678866+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:26.702069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.678865+0000","last_change":"2024-09-11T00:20:27.059553+0000","last_active":"2024-09-11T00:22:28.678865+0000","last_peered":"2024-09-11T00:22:28.678865+0000","last_clean":"2024-09-11T00:22:28.678865+0000","last_became_active":"2024-09-11T00:20:27.058979+0000","last_became_peered":"2024-09-11T00:20:27.058979+0000","last_unstale":"2024-09-11T00:22:28.678865+0000","last_undegraded":"2024-09-11T00:22:28.678865+0000","last_fullsized":"2024-09-11T00:22:28.678865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:21.472828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.831629+0000","last_change":"2024-09-11T00:22:28.831536+0000","last_active":"2024-09-11T00:22:28.831629+0000","last_peered":"2024-09-11T00:22:28.831629+0000","last_clean":"2024-09-11T00:22:28.831629+0000","last_became_active":"2024-09-11T00:22:28.831151+0000","last_became_peered":"2024-09-11T00:22:28.831151+0000","last_unstale":"2024-09-11T00:22:28.831629+0000","last_undegraded":"2024-09-11T00:22:28.831629+0000","last_fullsized":"2024-09-11T00:22:28.831629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.532570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864880+0000","last_change":"2024-09-11T00:20:27.059140+0000","last_active":"2024-09-11T00:22:28.864880+0000","last_peered":"2024-09-11T00:22:28.864880+0000","last_clean":"2024-09-11T00:22:28.864880+0000","last_became_active":"2024-09-11T00:20:27.058765+0000","last_became_peered":"2024-09-11T00:20:27.058765+0000","last_unstale":"2024-09-11T00:22:28.864880+0000","last_undegraded":"2024-09-11T00:22:28.864880+0000","last_fullsized":"2024-09-11T00:22:28.864880+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:43.947411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866106+0000","last_change":"2024-09-11T00:20:27.061149+0000","last_active":"2024-09-11T00:22:28.866106+0000","last_peered":"2024-09-11T00:22:28.866106+0000","last_clean":"2024-09-11T00:22:28.866106+0000","last_became_active":"2024-09-11T00:20:27.059949+0000","last_became_peered":"2024-09-11T00:20:27.059949+0000","last_unstale":"2024-09-11T00:22:28.866106+0000","last_undegraded":"2024-09-11T00:22:28.866106+0000","last_fullsized":"2024-09-11T00:22:28.866106+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:03.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830617+0000","last_change":"2024-09-11T00:21:28.231322+0000","last_active":"2024-09-11T00:22:28.830617+0000","last_peered":"2024-09-11T00:22:28.830617+0000","last_clean":"2024-09-11T00:22:28.830617+0000","last_became_active":"2024-09-11T00:21:28.231075+0000","last_became_peered":"2024-09-11T00:21:28.231075+0000","last_unstale":"2024-09-11T00:22:28.830617+0000","last_undegraded":"2024-09-11T00:22:28.830617+0000","last_fullsized":"2024-09-11T00:22:28.830617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:50.887001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865942+0000","last_change":"2024-09-11T00:20:27.058175+0000","last_active":"2024-09-11T00:22:28.865942+0000","last_peered":"2024-09-11T00:22:28.865942+0000","last_clean":"2024-09-11T00:22:28.865942+0000","last_became_active":"2024-09-11T00:20:27.057797+0000","last_became_peered":"2024-09-11T00:20:27.057797+0000","last_unstale":"2024-09-11T00:22:28.865942+0000","last_undegraded":"2024-09-11T00:22:28.865942+0000","last_fullsized":"2024-09-11T00:22:28.865942+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:29.370400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863468+0000","last_change":"2024-09-11T00:20:27.062024+0000","last_active":"2024-09-11T00:22:28.863468+0000","last_peered":"2024-09-11T00:22:28.863468+0000","last_clean":"2024-09-11T00:22:28.863468+0000","last_became_active":"2024-09-11T00:20:27.061763+0000","last_became_peered":"2024-09-11T00:20:27.061763+0000","last_unstale":"2024-09-11T00:22:28.863468+0000","last_undegraded":"2024-09-11T00:22:28.863468+0000","last_fullsized":"2024-09-11T00:22:28.863468+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:07.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865489+0000","last_change":"2024-09-11T00:20:27.060258+0000","last_active":"2024-09-11T00:22:28.865489+0000","last_peered":"2024-09-11T00:22:28.865489+0000","last_clean":"2024-09-11T00:22:28.865489+0000","last_became_active":"2024-09-11T00:20:27.059475+0000","last_became_peered":"2024-09-11T00:20:27.059475+0000","last_unstale":"2024-09-11T00:22:28.865489+0000","last_undegraded":"2024-09-11T00:22:28.865489+0000","last_fullsized":"2024-09-11T00:22:28.865489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:27.186322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830107+0000","last_change":"2024-09-11T00:20:27.059730+0000","last_active":"2024-09-11T00:22:28.830107+0000","last_peered":"2024-09-11T00:22:28.830107+0000","last_clean":"2024-09-11T00:22:28.830107+0000","last_became_active":"2024-09-11T00:20:27.059389+0000","last_became_peered":"2024-09-11T00:20:27.059389+0000","last_unstale":"2024-09-11T00:22:28.830107+0000","last_undegraded":"2024-09-11T00:22:28.830107+0000","last_fullsized":"2024-09-11T00:22:28.830107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864185+0000","last_change":"2024-09-11T00:20:27.072765+0000","last_active":"2024-09-11T00:22:28.864185+0000","last_peered":"2024-09-11T00:22:28.864185+0000","last_clean":"2024-09-11T00:22:28.864185+0000","last_became_active":"2024-09-11T00:20:27.072392+0000","last_became_peered":"2024-09-11T00:20:27.072392+0000","last_unstale":"2024-09-11T00:22:28.864185+0000","last_undegraded":"2024-09-11T00:22:28.864185+0000","last_fullsized":"2024-09-11T00:22:28.864185+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:27.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864725+0000","last_change":"2024-09-11T00:20:27.056575+0000","last_active":"2024-09-11T00:22:28.864725+0000","last_peered":"2024-09-11T00:22:28.864725+0000","last_clean":"2024-09-11T00:22:28.864725+0000","last_became_active":"2024-09-11T00:20:27.056264+0000","last_became_peered":"2024-09-11T00:20:27.056264+0000","last_unstale":"2024-09-11T00:22:28.864725+0000","last_undegraded":"2024-09-11T00:22:28.864725+0000","last_fullsized":"2024-09-11T00:22:28.864725+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:39.722518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679398+0000","last_change":"2024-09-11T00:20:27.065153+0000","last_active":"2024-09-11T00:22:28.679398+0000","last_peered":"2024-09-11T00:22:28.679398+0000","last_clean":"2024-09-11T00:22:28.679398+0000","last_became_active":"2024-09-11T00:20:27.062978+0000","last_became_peered":"2024-09-11T00:20:27.062978+0000","last_unstale":"2024-09-11T00:22:28.679398+0000","last_undegraded":"2024-09-11T00:22:28.679398+0000","last_fullsized":"2024-09-11T00:22:28.679398+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:37.820467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679290+0000","last_change":"2024-09-11T00:20:27.070085+0000","last_active":"2024-09-11T00:22:28.679290+0000","last_peered":"2024-09-11T00:22:28.679290+0000","last_clean":"2024-09-11T00:22:28.679290+0000","last_became_active":"2024-09-11T00:20:27.069809+0000","last_became_peered":"2024-09-11T00:20:27.069809+0000","last_unstale":"2024-09-11T00:22:28.679290+0000","last_undegraded":"2024-09-11T00:22:28.679290+0000","last_fullsized":"2024-09-11T00:22:28.679290+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:01.048590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864515+0000","last_change":"2024-09-11T00:20:27.064584+0000","last_active":"2024-09-11T00:22:28.864515+0000","last_peered":"2024-09-11T00:22:28.864515+0000","last_clean":"2024-09-11T00:22:28.864515+0000","last_became_active":"2024-09-11T00:20:27.064271+0000","last_became_peered":"2024-09-11T00:20:27.064271+0000","last_unstale":"2024-09-11T00:22:28.864515+0000","last_undegraded":"2024-09-11T00:22:28.864515+0000","last_fullsized":"2024-09-11T00:22:28.864515+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:53.717744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66073,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":443,"num_read_kb":292,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710760,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":515118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906988,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1712128,"data_stored":1164346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070514,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":519338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266744,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":515050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1634304,"data_stored":1111071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":31914445},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":510418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822729,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1642496,"data_stored":1106659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986255,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":515068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182484,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":510612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1110016,"data_stored":574141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":572400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1044480,"data_stored":511806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:22:38.979 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:22:38.979 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T00:22:38.979 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T00:22:38.979 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T00:22:39.354 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:39.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:39 smithi103 ceph-mon[30578]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:22:39.492 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:39 smithi103 ceph-mon[30578]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T00:22:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:39 smithi183 ceph-mon[35934]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:22:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:39 smithi183 ceph-mon[35934]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T00:22:39.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:39 smithi117 ceph-mon[35525]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:22:39.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:39 smithi117 ceph-mon[35525]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-09-11T00:22:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 -- 172.21.15.103:0/836092293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275010a820 msgr2=0x7f275010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:39.963 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 --2- 172.21.15.103:0/836092293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275010a820 0x7f275010ac60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f273c009900 tx=0x7f273c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 -- 172.21.15.103:0/836092293 shutdown_connections 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 --2- 172.21.15.103:0/836092293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f275010b1a0 0x7f27501119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 --2- 172.21.15.103:0/836092293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f275010a820 0x7f275010ac60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 --2- 172.21.15.103:0/836092293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2750104e10 0x7f27501051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 -- 172.21.15.103:0/836092293 >> 172.21.15.103:0/836092293 conn(0x7f2750100840 msgr2=0x7f2750102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 -- 172.21.15.103:0/836092293 shutdown_connections 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.955+0000 7f2756a00640 1 -- 172.21.15.103:0/836092293 wait complete. 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 Processor -- start 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 -- start start 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750104e10 0x7f275019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f275010a820 0x7f275019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f275010b1a0 0x7f27501a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:39.964 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2750114e60 con 0x7f2750104e10 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f2750114ce0 con 0x7f275010b1a0 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.956+0000 7f2756a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2750114fe0 con 0x7f275010a820 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750104e10 0x7f275019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2756000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f275010b1a0 0x7f27501a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750104e10 0x7f275019b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41008/0 (socket says 172.21.15.103:41008) 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 -- 172.21.15.103:0/4039407944 learned_addr learned my addr 172.21.15.103:0/4039407944 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2754c00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f275010a820 0x7f275019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 -- 172.21.15.103:0/4039407944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f275010a820 msgr2=0x7f275019b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f275010a820 0x7f275019b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 -- 172.21.15.103:0/4039407944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f275010b1a0 msgr2=0x7f27501a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f275010b1a0 0x7f27501a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:39.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 -- 172.21.15.103:0/4039407944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27501a12e0 con 0x7f2750104e10 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2756000640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f275010b1a0 0x7f27501a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.957+0000 7f2755600640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750104e10 0x7f275019b230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f274400eb90 tx=0x7f274400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.958+0000 7f274e000640 1 -- 172.21.15.103:0/4039407944 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744004d20 con 0x7f2750104e10 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.958+0000 7f274e000640 1 -- 172.21.15.103:0/4039407944 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f274400ee90 con 0x7f2750104e10 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.958+0000 7f274e000640 1 -- 172.21.15.103:0/4039407944 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744010670 con 0x7f2750104e10 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.958+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27501a15d0 con 0x7f2750104e10 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.958+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27501a7c90 con 0x7f2750104e10 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.959+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27501051f0 con 0x7f2750104e10 2024-09-11T00:22:39.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.961+0000 7f274e000640 1 -- 172.21.15.103:0/4039407944 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f2744010870 con 0x7f2750104e10 2024-09-11T00:22:39.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.962+0000 7f274e000640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27380b5490 0x7f27380b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:39.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.963+0000 7f274e000640 1 -- 172.21.15.103:0/4039407944 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f27440e68c0 con 0x7f2750104e10 2024-09-11T00:22:39.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.963+0000 7f2754c00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27380b5490 0x7f27380b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:39.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.964+0000 7f2754c00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27380b5490 0x7f27380b7950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f273c031440 tx=0x7f273c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:39.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:39.965+0000 7f274e000640 1 -- 172.21.15.103:0/4039407944 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2744056220 con 0x7f2750104e10 2024-09-11T00:22:40.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.189+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f275019c360 con 0x7f2750104e10 2024-09-11T00:22:40.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.189+0000 7f274e000640 1 -- 172.21.15.103:0/4039407944 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f27440ad230 con 0x7f2750104e10 2024-09-11T00:22:40.191 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:22:40.191 INFO:teuthology.orchestra.run.smithi103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27380b5490 msgr2=0x7f27380b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27380b5490 0x7f27380b7950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f273c031440 tx=0x7f273c005b50 comp rx=0 tx=0).stop 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750104e10 msgr2=0x7f275019b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750104e10 0x7f275019b230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f274400eb90 tx=0x7f274400c6a0 comp rx=0 tx=0).stop 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 shutdown_connections 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f27380b5490 0x7f27380b7950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f275010b1a0 0x7f27501a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:40.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f275010a820 0x7f275019b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:40.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 --2- 172.21.15.103:0/4039407944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2750104e10 0x7f275019b230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:40.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.191+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 >> 172.21.15.103:0/4039407944 conn(0x7f2750100840 msgr2=0x7f2750102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:40.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.192+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 shutdown_connections 2024-09-11T00:22:40.195 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:40.192+0000 7f2756a00640 1 -- 172.21.15.103:0/4039407944 wait complete. 2024-09-11T00:22:40.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:40 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4039407944' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:22:40.593 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:40 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4039407944' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:22:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:40 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4039407944' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-11T00:22:41.059 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T00:22:41.059 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:22:41.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:41 smithi103 ceph-mon[30578]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-11T00:22:41.433 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:41 smithi183 ceph-mon[35934]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-11T00:22:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:41 smithi117 ceph-mon[35525]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-11T00:22:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 -- 172.21.15.103:0/1373196168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a810f760 msgr2=0x7fe5a8115f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 --2- 172.21.15.103:0/1373196168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a810f760 0x7fe5a8115f60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009ca0 tx=0x7fe5a002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 -- 172.21.15.103:0/1373196168 shutdown_connections 2024-09-11T00:22:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 --2- 172.21.15.103:0/1373196168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a810f760 0x7fe5a8115f60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 --2- 172.21.15.103:0/1373196168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5a810ede0 0x7fe5a810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 --2- 172.21.15.103:0/1373196168 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a81093d0 0x7fe5a81097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 -- 172.21.15.103:0/1373196168 >> 172.21.15.103:0/1373196168 conn(0x7fe5a8078a40 msgr2=0x7fe5a807ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.054+0000 7fe5ae400640 1 -- 172.21.15.103:0/1373196168 shutdown_connections 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.055+0000 7fe5ae400640 1 -- 172.21.15.103:0/1373196168 wait complete. 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.055+0000 7fe5ae400640 1 Processor -- start 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5ae400640 1 -- start start 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5ae400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81093d0 0x7fe5a819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5ae400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a810ede0 0x7fe5a819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5ae400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5a810f760 0x7fe5a81a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5ae400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8119390 con 0x7fe5a810f760 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5ae400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8119210 con 0x7fe5a81093d0 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5ae400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a8119510 con 0x7fe5a810ede0 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81093d0 0x7fe5a819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:42.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5a7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5a810f760 0x7fe5a81a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5a7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81093d0 0x7fe5a819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:32810/0 (socket says 172.21.15.103:32810) 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5a7400640 1 -- 172.21.15.103:0/960814963 learned_addr learned my addr 172.21.15.103:0/960814963 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.056+0000 7fe5a6a00640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a810ede0 0x7fe5a819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5a7400640 1 -- 172.21.15.103:0/960814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a810ede0 msgr2=0x7fe5a819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5a7400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a810ede0 0x7fe5a819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5a7400640 1 -- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5a810f760 msgr2=0x7fe5a81a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5a7400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5a810f760 0x7fe5a81a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5a7400640 1 -- 172.21.15.103:0/960814963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a81a58d0 con 0x7fe5a81093d0 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5a7400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81093d0 0x7fe5a819f820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe59fe00640 1 -- 172.21.15.103:0/960814963 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59800cd80 con 0x7fe5a81093d0 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe59fe00640 1 -- 172.21.15.103:0/960814963 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5980054d0 con 0x7fe5a81093d0 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe59fe00640 1 -- 172.21.15.103:0/960814963 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598010690 con 0x7fe5a81093d0 2024-09-11T00:22:42.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a81a5bc0 con 0x7fe5a81093d0 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.057+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a81ac280 con 0x7fe5a81093d0 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.058+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a81097b0 con 0x7fe5a81093d0 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.061+0000 7fe59fe00640 1 -- 172.21.15.103:0/960814963 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fe598004e30 con 0x7fe5a81093d0 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.062+0000 7fe59fe00640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe58c0b53f0 0x7fe58c0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.063+0000 7fe59fe00640 1 -- 172.21.15.103:0/960814963 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe5980e66f0 con 0x7fe5a81093d0 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.063+0000 7fe5a6a00640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe58c0b53f0 0x7fe58c0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.063+0000 7fe5a6a00640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe58c0b53f0 0x7fe58c0b78b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe5940095f0 tx=0x7fe594005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:42.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.064+0000 7fe59fe00640 1 -- 172.21.15.103:0/960814963 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe598056220 con 0x7fe5a81093d0 2024-09-11T00:22:42.284 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.282+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5a81ac630 con 0x7fe5a81093d0 2024-09-11T00:22:42.285 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.283+0000 7fe59fe00640 1 -- 172.21.15.103:0/960814963 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7fe5980ad060 con 0x7fe5a81093d0 2024-09-11T00:22:42.285 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:22:42.286 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-11T00:22:42.286 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-11T00:21:28:210497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:21:27.247376+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:22:42.288 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.286+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe58c0b53f0 msgr2=0x7fe58c0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.286+0000 7fe5ae400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe58c0b53f0 0x7fe58c0b78b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe5940095f0 tx=0x7fe594005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.286+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81093d0 msgr2=0x7fe5a819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.286+0000 7fe5ae400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81093d0 0x7fe5a819f820 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.286+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 shutdown_connections 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.287+0000 7fe5ae400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe58c0b53f0 0x7fe58c0b78b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.287+0000 7fe5ae400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5a810f760 0x7fe5a81a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.287+0000 7fe5ae400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5a810ede0 0x7fe5a819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.287+0000 7fe5ae400640 1 --2- 172.21.15.103:0/960814963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5a81093d0 0x7fe5a819f820 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.287+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 >> 172.21.15.103:0/960814963 conn(0x7fe5a8078a40 msgr2=0x7fe5a807ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.287+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 shutdown_connections 2024-09-11T00:22:42.289 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:42.287+0000 7fe5ae400640 1 -- 172.21.15.103:0/960814963 wait complete. 2024-09-11T00:22:42.588 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:42 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/960814963' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:42.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:42 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/960814963' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:42 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/960814963' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:43.126 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:22:43.500 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:43.639 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:43 smithi103 ceph-mon[30578]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:43.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:43 smithi117 ceph-mon[35525]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:43.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:43 smithi183 ceph-mon[35934]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:44.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.124+0000 7ff402000640 1 -- 172.21.15.103:0/313705601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc10cf60 msgr2=0x7ff3fc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:44.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.124+0000 7ff402000640 1 --2- 172.21.15.103:0/313705601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc10cf60 0x7ff3fc10f3c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009ca0 tx=0x7ff3f402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.124+0000 7ff402000640 1 -- 172.21.15.103:0/313705601 shutdown_connections 2024-09-11T00:22:44.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.124+0000 7ff402000640 1 --2- 172.21.15.103:0/313705601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc10cf60 0x7ff3fc10f3c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.124+0000 7ff402000640 1 --2- 172.21.15.103:0/313705601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc104420 0x7ff3fc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.124+0000 7ff402000640 1 --2- 172.21.15.103:0/313705601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3fc103b00 0x7ff3fc103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.124+0000 7ff402000640 1 -- 172.21.15.103:0/313705601 >> 172.21.15.103:0/313705601 conn(0x7ff3fc0fd560 msgr2=0x7ff3fc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:44.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.125+0000 7ff402000640 1 -- 172.21.15.103:0/313705601 shutdown_connections 2024-09-11T00:22:44.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.125+0000 7ff402000640 1 -- 172.21.15.103:0/313705601 wait complete. 2024-09-11T00:22:44.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.125+0000 7ff402000640 1 Processor -- start 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff402000640 1 -- start start 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff402000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc103b00 0x7ff3fc19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff402000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc104420 0x7ff3fc19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff402000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3fc10cf60 0x7ff3fc1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff402000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112c20 con 0x7ff3fc104420 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff402000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112aa0 con 0x7ff3fc10cf60 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff402000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112da0 con 0x7ff3fc103b00 2024-09-11T00:22:44.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc104420 0x7ff3fc19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc104420 0x7ff3fc19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35854/0 (socket says 172.21.15.103:35854) 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff3fbe00640 1 -- 172.21.15.103:0/1029761681 learned_addr learned my addr 172.21.15.103:0/1029761681 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff401600640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3fc10cf60 0x7ff3fc1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.126+0000 7ff400c00640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc103b00 0x7ff3fc19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3fbe00640 1 -- 172.21.15.103:0/1029761681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc103b00 msgr2=0x7ff3fc19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3fbe00640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc103b00 0x7ff3fc19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3fbe00640 1 -- 172.21.15.103:0/1029761681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3fc10cf60 msgr2=0x7ff3fc1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3fbe00640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3fc10cf60 0x7ff3fc1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3fbe00640 1 -- 172.21.15.103:0/1029761681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc1a1340 con 0x7ff3fc104420 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff401600640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3fc10cf60 0x7ff3fc1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3fbe00640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc104420 0x7ff3fc19b7d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800e9c0 tx=0x7ff3e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:44.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3f9600640 1 -- 172.21.15.103:0/1029761681 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800ccc0 con 0x7ff3fc104420 2024-09-11T00:22:44.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3f9600640 1 -- 172.21.15.103:0/1029761681 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e80035f0 con 0x7ff3fc104420 2024-09-11T00:22:44.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.127+0000 7ff3f9600640 1 -- 172.21.15.103:0/1029761681 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800ce60 con 0x7ff3fc104420 2024-09-11T00:22:44.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.128+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc1a1630 con 0x7ff3fc104420 2024-09-11T00:22:44.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.128+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc1a7cf0 con 0x7ff3fc104420 2024-09-11T00:22:44.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.129+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3fc103ee0 con 0x7ff3fc104420 2024-09-11T00:22:44.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.131+0000 7ff3f9600640 1 -- 172.21.15.103:0/1029761681 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7ff3e8004000 con 0x7ff3fc104420 2024-09-11T00:22:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.132+0000 7ff3f9600640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff3e40b5410 0x7ff3e40b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.132+0000 7ff3f9600640 1 -- 172.21.15.103:0/1029761681 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff3e80e64c0 con 0x7ff3fc104420 2024-09-11T00:22:44.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.133+0000 7ff400c00640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff3e40b5410 0x7ff3e40b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:44.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.133+0000 7ff400c00640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff3e40b5410 0x7ff3e40b78d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009bd0 tx=0x7ff3f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:44.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.134+0000 7ff3f9600640 1 -- 172.21.15.103:0/1029761681 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff3e80ace30 con 0x7ff3fc104420 2024-09-11T00:22:44.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.356+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3fc1a80e0 con 0x7ff3fc104420 2024-09-11T00:22:44.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.357+0000 7ff3f9600640 1 -- 172.21.15.103:0/1029761681 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7ff3e80ac730 con 0x7ff3fc104420 2024-09-11T00:22:44.359 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:22:44.359 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-11T00:21:28:210497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:21:27.247376+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:22:44.359 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-11T00:22:44.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff3e40b5410 msgr2=0x7ff3e40b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:44.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff3e40b5410 0x7ff3e40b78d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009bd0 tx=0x7ff3f0005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc104420 msgr2=0x7ff3fc19b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc104420 0x7ff3fc19b7d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800e9c0 tx=0x7ff3e800ee80 comp rx=0 tx=0).stop 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 shutdown_connections 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff3e40b5410 0x7ff3e40b78d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3fc10cf60 0x7ff3fc1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc104420 0x7ff3fc19b7d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 --2- 172.21.15.103:0/1029761681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff3fc103b00 0x7ff3fc19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.359+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 >> 172.21.15.103:0/1029761681 conn(0x7ff3fc0fd560 msgr2=0x7ff3fc0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.360+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 shutdown_connections 2024-09-11T00:22:44.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:44.360+0000 7ff402000640 1 -- 172.21.15.103:0/1029761681 wait complete. 2024-09-11T00:22:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:44 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1029761681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:44.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:44 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1029761681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:44 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1029761681' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:45.145 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:22:45.519 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:45.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:45 smithi103 ceph-mon[30578]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:45.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:45 smithi117 ceph-mon[35525]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:45.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:45 smithi183 ceph-mon[35934]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 -- 172.21.15.103:0/424889995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d84104fa0 msgr2=0x7f5d84105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 --2- 172.21.15.103:0/424889995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d84104fa0 0x7f5d84105380 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009900 tx=0x7f5d7402a410 comp rx=0 tx=0).stop 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 -- 172.21.15.103:0/424889995 shutdown_connections 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 --2- 172.21.15.103:0/424889995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8410b330 0x7f5d84111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 --2- 172.21.15.103:0/424889995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d8410a9b0 0x7f5d8410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 --2- 172.21.15.103:0/424889995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d84104fa0 0x7f5d84105380 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 -- 172.21.15.103:0/424889995 >> 172.21.15.103:0/424889995 conn(0x7f5d84100a50 msgr2=0x7f5d84102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 -- 172.21.15.103:0/424889995 shutdown_connections 2024-09-11T00:22:46.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.100+0000 7f5d8b000640 1 -- 172.21.15.103:0/424889995 wait complete. 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.101+0000 7f5d8b000640 1 Processor -- start 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.101+0000 7f5d8b000640 1 -- start start 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d8b000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d84104fa0 0x7f5d8419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d8b000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8410a9b0 0x7f5d8419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d8b000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8410b330 0x7f5d841a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d8b000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84114fb0 con 0x7f5d8410a9b0 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d8b000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84114e30 con 0x7f5d84104fa0 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d8b000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84115130 con 0x7f5d8410b330 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8410a9b0 0x7f5d8419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8410a9b0 0x7f5d8419ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35866/0 (socket says 172.21.15.103:35866) 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 -- 172.21.15.103:0/819220069 learned_addr learned my addr 172.21.15.103:0/819220069 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d83e00640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d84104fa0 0x7f5d8419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:46.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d89000640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8410b330 0x7f5d841a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 -- 172.21.15.103:0/819220069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8410b330 msgr2=0x7f5d841a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8410b330 0x7f5d841a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 -- 172.21.15.103:0/819220069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d84104fa0 msgr2=0x7f5d8419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d84104fa0 0x7f5d8419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.102+0000 7f5d7be00640 1 -- 172.21.15.103:0/819220069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d841a1570 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.103+0000 7f5d7be00640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8410a9b0 0x7f5d8419ba00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00e9f0 tx=0x7f5d6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.103+0000 7f5d81600640 1 -- 172.21.15.103:0/819220069 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00cd80 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.103+0000 7f5d81600640 1 -- 172.21.15.103:0/819220069 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d6c0108e0 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.103+0000 7f5d81600640 1 -- 172.21.15.103:0/819220069 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c018980 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.103+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d841a1860 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.103+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a7f20 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.104+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d84105380 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.106+0000 7f5d81600640 1 -- 172.21.15.103:0/819220069 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f5d6c010430 con 0x7f5d8410a9b0 2024-09-11T00:22:46.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.108+0000 7f5d81600640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d680b53f0 0x7f5d680b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:46.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.108+0000 7f5d81600640 1 -- 172.21.15.103:0/819220069 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5d6c0e63b0 con 0x7f5d8410a9b0 2024-09-11T00:22:46.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.108+0000 7f5d83e00640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d680b53f0 0x7f5d680b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:46.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.109+0000 7f5d83e00640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d680b53f0 0x7f5d680b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005ab0 tx=0x7f5d74005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:46.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.112+0000 7f5d81600640 1 -- 172.21.15.103:0/819220069 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5d6c0ac6d0 con 0x7f5d8410a9b0 2024-09-11T00:22:46.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.300+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d8402e5c0 con 0x7f5d8410a9b0 2024-09-11T00:22:46.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.301+0000 7f5d81600640 1 -- 172.21.15.103:0/819220069 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24865 (secure 0 0 0) 0x7f5d6c0b1100 con 0x7f5d8410a9b0 2024-09-11T00:22:46.303 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:22:46.304 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":89,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:22:28.671420+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.303+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d680b53f0 msgr2=0x7f5d680b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d680b53f0 0x7f5d680b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5d74005ab0 tx=0x7f5d74005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8410a9b0 msgr2=0x7f5d8419ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8410a9b0 0x7f5d8419ba00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c00e9f0 tx=0x7f5d6c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 shutdown_connections 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d680b53f0 0x7f5d680b78b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d8410b330 0x7f5d841a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d8410a9b0 0x7f5d8419ba00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 --2- 172.21.15.103:0/819220069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d84104fa0 0x7f5d8419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 >> 172.21.15.103:0/819220069 conn(0x7f5d84100a50 msgr2=0x7f5d84101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 shutdown_connections 2024-09-11T00:22:46.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:46.304+0000 7f5d8b000640 1 -- 172.21.15.103:0/819220069 wait complete. 2024-09-11T00:22:46.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:46 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/819220069' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:22:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:46 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/819220069' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:22:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:46 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/819220069' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:22:47.188 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:22:47.577 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:47.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:47 smithi117 ceph-mon[35525]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:47.716 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:47 smithi103 ceph-mon[30578]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:47.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:47 smithi183 ceph-mon[35934]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 -- 172.21.15.103:0/3576761471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490113830 msgr2=0x7fd490115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 --2- 172.21.15.103:0/3576761471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490113830 0x7fd490115c20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd488009be0 tx=0x7fd48802a6c0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 -- 172.21.15.103:0/3576761471 shutdown_connections 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 --2- 172.21.15.103:0/3576761471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490113830 0x7fd490115c20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 --2- 172.21.15.103:0/3576761471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd490110f00 0x7fd4901132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 --2- 172.21.15.103:0/3576761471 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490101220 0x7fd490101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 -- 172.21.15.103:0/3576761471 >> 172.21.15.103:0/3576761471 conn(0x7fd490078100 msgr2=0x7fd4900ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:48.211 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 -- 172.21.15.103:0/3576761471 shutdown_connections 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.170+0000 7fd498400640 1 -- 172.21.15.103:0/3576761471 wait complete. 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.171+0000 7fd498400640 1 Processor -- start 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd498400640 1 -- start start 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd498400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490101220 0x7fd49019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd498400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490110f00 0x7fd49019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd498400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd490113830 0x7fd4901a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd498400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4901193a0 con 0x7fd490113830 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd498400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd490119220 con 0x7fd490101220 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd498400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd490119520 con 0x7fd490110f00 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd495000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490110f00 0x7fd49019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd496400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd490113830 0x7fd4901a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:48.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd495000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490110f00 0x7fd49019fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:52754/0 (socket says 172.21.15.103:52754) 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd495000640 1 -- 172.21.15.103:0/3527182051 learned_addr learned my addr 172.21.15.103:0/3527182051 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.172+0000 7fd495a00640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490101220 0x7fd49019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd495000640 1 -- 172.21.15.103:0/3527182051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490101220 msgr2=0x7fd49019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd495000640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490101220 0x7fd49019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd495000640 1 -- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd490113830 msgr2=0x7fd4901a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd495000640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd490113830 0x7fd4901a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd495000640 1 -- 172.21.15.103:0/3527182051 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4901a5960 con 0x7fd490110f00 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd495000640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490110f00 0x7fd49019fdf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e9a0 tx=0x7fd47c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd48e000640 1 -- 172.21.15.103:0/3527182051 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00cd80 con 0x7fd490110f00 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd48e000640 1 -- 172.21.15.103:0/3527182051 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c0035f0 con 0x7fd490110f00 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4901a5c50 con 0x7fd490110f00 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.173+0000 7fd48e000640 1 -- 172.21.15.103:0/3527182051 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c010690 con 0x7fd490110f00 2024-09-11T00:22:48.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.174+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4901ac310 con 0x7fd490110f00 2024-09-11T00:22:48.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.174+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd490101600 con 0x7fd490110f00 2024-09-11T00:22:48.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.177+0000 7fd48e000640 1 -- 172.21.15.103:0/3527182051 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fd47c004000 con 0x7fd490110f00 2024-09-11T00:22:48.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.178+0000 7fd48e000640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4780b5370 0x7fd4780b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:48.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.179+0000 7fd48e000640 1 -- 172.21.15.103:0/3527182051 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd47c0e6410 con 0x7fd490110f00 2024-09-11T00:22:48.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.179+0000 7fd495a00640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4780b5370 0x7fd4780b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:48.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.179+0000 7fd495a00640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4780b5370 0x7fd4780b7830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd484009bd0 tx=0x7fd484005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:48.214 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.180+0000 7fd48e000640 1 -- 172.21.15.103:0/3527182051 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd47c0ace70 con 0x7fd490110f00 2024-09-11T00:22:48.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.402+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4901ac6e0 con 0x7fd490110f00 2024-09-11T00:22:48.404 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.403+0000 7fd48e000640 1 -- 172.21.15.103:0/3527182051 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7fd47c0ac770 con 0x7fd490110f00 2024-09-11T00:22:48.404 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:22:48.404 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-11T00:21:28:210497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:21:27.247376+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:22:48.405 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4780b5370 msgr2=0x7fd4780b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4780b5370 0x7fd4780b7830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd484009bd0 tx=0x7fd484005f90 comp rx=0 tx=0).stop 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490110f00 msgr2=0x7fd49019fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490110f00 0x7fd49019fdf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd47c00e9a0 tx=0x7fd47c00ee60 comp rx=0 tx=0).stop 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 shutdown_connections 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4780b5370 0x7fd4780b7830 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd490113830 0x7fd4901a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd490110f00 0x7fd49019fdf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 --2- 172.21.15.103:0/3527182051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd490101220 0x7fd49019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 >> 172.21.15.103:0/3527182051 conn(0x7fd490078100 msgr2=0x7fd490114050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:48.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 shutdown_connections 2024-09-11T00:22:48.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:48.406+0000 7fd498400640 1 -- 172.21.15.103:0/3527182051 wait complete. 2024-09-11T00:22:48.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:48 smithi103 ceph-mon[30578]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:48.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:48 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3527182051' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:48 smithi117 ceph-mon[35525]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:48 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3527182051' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:48 smithi183 ceph-mon[35934]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:22:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:48 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3527182051' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:22:49.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:20:28.229329+0000', 'modified': '2024-09-11T00:21:27.247376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24656, 'mds_1': 24643, 'mds_2': 24628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24599': {'gid': 24599, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/2973217634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2973217634}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2973217634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6837/2700270511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 2700270511}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2700270511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24622': {'gid': 24622, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/3946403859', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 3946403859}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3946403859}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24628': {'gid': 24628, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.183:6837/335217865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 335217865}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 335217865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24643': {'gid': 24643, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.183:6839/424027451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 424027451}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 424027451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/3769588373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 3769588373}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 3769588373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24656, 'qdb_cluster': [24656, 24643, 24628]} 2024-09-11T00:22:49.301 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T00:22:49.302 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T00:22:49.673 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 -- 172.21.15.103:0/3765398799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 msgr2=0x7f49ac10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 --2- 172.21.15.103:0/3765398799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 0x7f49ac10ade0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4994009900 tx=0x7f499402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 -- 172.21.15.103:0/3765398799 shutdown_connections 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 --2- 172.21.15.103:0/3765398799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac10b320 0x7f49ac111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 --2- 172.21.15.103:0/3765398799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 0x7f49ac10ade0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 --2- 172.21.15.103:0/3765398799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac104f90 0x7f49ac105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 -- 172.21.15.103:0/3765398799 >> 172.21.15.103:0/3765398799 conn(0x7f49ac100a00 msgr2=0x7f49ac102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:50.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 -- 172.21.15.103:0/3765398799 shutdown_connections 2024-09-11T00:22:50.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.340+0000 7f49b2e00640 1 -- 172.21.15.103:0/3765398799 wait complete. 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 Processor -- start 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 -- start start 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac104f90 0x7f49ac0776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 0x7f49ac077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac10b320 0x7f49ac078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac114fa0 con 0x7f49ac10a9a0 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac114e20 con 0x7f49ac10b320 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49b2e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac115120 con 0x7f49ac104f90 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.341+0000 7f49abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac104f90 0x7f49ac0776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 0x7f49ac077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac104f90 0x7f49ac0776b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:52766/0 (socket says 172.21.15.103:52766) 2024-09-11T00:22:50.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 -- 172.21.15.103:0/2528778443 learned_addr learned my addr 172.21.15.103:0/2528778443 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:50.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49b0e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac10b320 0x7f49ac078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 -- 172.21.15.103:0/2528778443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac10b320 msgr2=0x7f49ac078130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac10b320 0x7f49ac078130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 -- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 msgr2=0x7f49ac077bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 0x7f49ac077bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 -- 172.21.15.103:0/2528778443 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac1abaa0 con 0x7f49ac104f90 2024-09-11T00:22:50.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.342+0000 7f49abe00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac104f90 0x7f49ac0776b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d950 tx=0x7f499c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:50.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.343+0000 7f49a9600640 1 -- 172.21.15.103:0/2528778443 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c017030 con 0x7f49ac104f90 2024-09-11T00:22:50.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.343+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac1abd90 con 0x7f49ac104f90 2024-09-11T00:22:50.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.343+0000 7f49a9600640 1 -- 172.21.15.103:0/2528778443 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c004020 con 0x7f49ac104f90 2024-09-11T00:22:50.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.343+0000 7f49a9600640 1 -- 172.21.15.103:0/2528778443 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0047a0 con 0x7f49ac104f90 2024-09-11T00:22:50.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.343+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac1ac250 con 0x7f49ac104f90 2024-09-11T00:22:50.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.346+0000 7f49b5a23740 1 Processor -- start 2024-09-11T00:22:50.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.346+0000 7f49b5a23740 1 -- start start 2024-09-11T00:22:50.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.346+0000 7f49a9600640 1 -- 172.21.15.103:0/2528778443 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f499c009a30 con 0x7f49ac104f90 2024-09-11T00:22:50.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198a240 0x55cbf198a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198ab60 0x55cbf19965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf1998ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49b5a23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cbf1901d50 con 0x55cbf198a240 2024-09-11T00:22:50.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49b5a23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbf18f1180 con 0x55cbf198ab60 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49b5a23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cbf18f4f90 con 0x55cbf1996af0 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198ab60 0x55cbf19965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49b0e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198a240 0x55cbf198a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198ab60 0x55cbf19965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:32878/0 (socket says 172.21.15.103:32878) 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 -- 172.21.15.103:0/2625431270 learned_addr learned my addr 172.21.15.103:0/2625431270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49a3e00640 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf1998ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 -- 172.21.15.103:0/2625431270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 msgr2=0x55cbf1998ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf1998ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 -- 172.21.15.103:0/2625431270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198a240 msgr2=0x55cbf198a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198a240 0x55cbf198a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.347+0000 7f49abe00640 1 -- 172.21.15.103:0/2625431270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf18bba70 con 0x55cbf198ab60 2024-09-11T00:22:50.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.348+0000 7f49abe00640 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198ab60 0x55cbf19965b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f499c0e9b60 tx=0x7f499c0ee640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c406661e701e79e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:50.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.348+0000 7f49a1600640 1 -- 172.21.15.103:0/2625431270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0f5020 con 0x55cbf198ab60 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.348+0000 7f49a1600640 1 -- 172.21.15.103:0/2625431270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c0f18a0 con 0x55cbf198ab60 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.348+0000 7f49a1600640 1 -- 172.21.15.103:0/2625431270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0f0550 con 0x55cbf198ab60 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.348+0000 7f49a9600640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49900b5370 0x7f49900b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49a9600640 1 -- 172.21.15.103:0/2528778443 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f499c0e5b80 con 0x7f49ac104f90 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 -- 172.21.15.103:0/2625431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198ab60 msgr2=0x55cbf19965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198ab60 0x55cbf19965b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f499c0e9b60 tx=0x7f499c0ee640 comp rx=0 tx=0).stop 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b0e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49900b5370 0x7f49900b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 -- 172.21.15.103:0/2625431270 shutdown_connections 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf1998ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198ab60 0x55cbf19965b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2625431270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198a240 0x55cbf198a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 -- 172.21.15.103:0/2625431270 >> 172.21.15.103:0/2625431270 conn(0x55cbf1988020 msgr2=0x55cbf1989210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:50.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 -- 172.21.15.103:0/2625431270 shutdown_connections 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b5a23740 1 -- 172.21.15.103:0/2625431270 wait complete. 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.349+0000 7f49b0e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49900b5370 0x7f49900b7830 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49a4005fd0 tx=0x7f49a4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 -- start start 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 Processor -- start 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf1993a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198ab60 0x55cbf1993f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf1996af0 0x55cbf19944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cbf17525f0 con 0x55cbf1996af0 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbf188c460 con 0x55cbf198a240 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cbf1999840 con 0x55cbf198ab60 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.350+0000 7f49b5a23740 10 client.0 _ll_register_callbacks cb 0x55cbf189c210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf1993a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf1993a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:32888/0 (socket says 172.21.15.103:32888) 2024-09-11T00:22:50.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 -- 172.21.15.103:0/767954616 learned_addr learned my addr 172.21.15.103:0/767954616 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:50.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49a3e00640 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198ab60 0x55cbf1993f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49b0e00640 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf1996af0 0x55cbf19944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 -- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198ab60 msgr2=0x55cbf1993f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198ab60 0x55cbf1993f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 -- 172.21.15.103:0/767954616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf1996af0 msgr2=0x55cbf19944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf1996af0 0x55cbf19944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 -- 172.21.15.103:0/767954616 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf164f540 con 0x55cbf198a240 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f49abe00640 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf1993a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f499c0eeb40 tx=0x7f499c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9baec6d7704abc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.351+0000 7f499a000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:22:50.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f499a000640 1 -- 172.21.15.103:0/767954616 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c0f5070 con 0x55cbf198a240 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f499a000640 1 -- 172.21.15.103:0/767954616 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f499c093110 con 0x55cbf198a240 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbf188ea70 con 0x55cbf198a240 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f499a000640 1 -- 172.21.15.103:0/767954616 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f499c0a8440 con 0x55cbf198a240 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f49b5a23740 10 client.25100 fetch_fsmap learned FSMap version 26 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbf18a96f0 con 0x55cbf198a240 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f499a000640 1 -- 172.21.15.103:0/767954616 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cbf188ea70 con 0x55cbf198a240 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f499a000640 1 -- 172.21.15.103:0/767954616 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f499c0e3390 con 0x55cbf198a240 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f49b5a23740 10 client.25100 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.352+0000 7f49b5a23740 10 client.25100 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.353+0000 7f49b5a23740 20 client.25100 populate_metadata read hostname 'smithi103' 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.353+0000 7f49b5a23740 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf19a7320 0x55cbf19a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.353+0000 7f49b5a23740 4 client.25100 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.353+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 --> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf1903ca0 con 0x55cbf19a7320 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.353+0000 7f49a3e00640 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf19a7320 0x55cbf19a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.354+0000 7f49a3e00640 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf19a7320 0x55cbf19a8790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73d0737fa55d6065 server_cookie=a854ca340bfdc8f in_seq=0 out_seq=0 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.354+0000 7f499a000640 10 client.25100 ms_handle_connect on v2:172.21.15.183:6836/335217865 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f499a000640 1 -- 172.21.15.103:0/767954616 <== mds.2 v2:172.21.15.183:6836/335217865 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cbf1903ca0 con 0x55cbf19a7320 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f499a000640 10 client.25100 handle_command_reply: tid=0 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 client.25100 shutdown 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 msgr2=0x55cbf1993a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf1993a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f499c0eeb40 tx=0x7f499c004730 comp rx=0 tx=0).stop 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 shutdown_connections 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf19a7320 0x55cbf19a8790 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf1996af0 0x55cbf19944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198ab60 0x55cbf1993f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 --2- 172.21.15.103:0/767954616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf1993a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f499a000640 0 client.25100 ms_handle_reset on v2:172.21.15.183:6836/335217865 2024-09-11T00:22:50.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 >> 172.21.15.103:0/767954616 conn(0x55cbf1988020 msgr2=0x55cbf198dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:50.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 shutdown_connections 2024-09-11T00:22:50.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 1 -- 172.21.15.103:0/767954616 wait complete. 2024-09-11T00:22:50.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.355+0000 7f49b5a23740 20 client.25100 trim_cache size 0 max 16384 2024-09-11T00:22:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 Processor -- start 2024-09-11T00:22:50.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 -- start start 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf19947e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf198e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf19a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cbf19a6ad0 con 0x55cbf198ab60 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbf19a71a0 con 0x55cbf198a240 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49b5a23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cbf19a7870 con 0x55cbf1996af0 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf198e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf198e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35942/0 (socket says 172.21.15.103:35942) 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49a3e00640 1 -- 172.21.15.103:0/2643496175 learned_addr learned my addr 172.21.15.103:0/2643496175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:50.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.374+0000 7f49abe00640 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf19947e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49b0e00640 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf19a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a3e00640 1 -- 172.21.15.103:0/2643496175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 msgr2=0x55cbf19a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a3e00640 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf19a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a3e00640 1 -- 172.21.15.103:0/2643496175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 msgr2=0x55cbf19947e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a3e00640 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf19947e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a3e00640 1 -- 172.21.15.103:0/2643496175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf19a7d20 con 0x55cbf198ab60 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49b0e00640 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf19a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:50.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a3e00640 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf198e1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4994006010 tx=0x7f4994005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cacb768abc7c840a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a0c00640 1 -- 172.21.15.103:0/2643496175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994035040 con 0x55cbf198ab60 2024-09-11T00:22:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a0c00640 1 -- 172.21.15.103:0/2643496175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4994004770 con 0x55cbf198ab60 2024-09-11T00:22:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.375+0000 7f49a0c00640 1 -- 172.21.15.103:0/2643496175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994003850 con 0x55cbf198ab60 2024-09-11T00:22:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 -- 172.21.15.103:0/2643496175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 msgr2=0x55cbf198e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf198e1a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4994006010 tx=0x7f4994005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 -- 172.21.15.103:0/2643496175 shutdown_connections 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf1996af0 0x55cbf19a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf198e1a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 --2- 172.21.15.103:0/2643496175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf198a240 0x55cbf19947e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 -- 172.21.15.103:0/2643496175 >> 172.21.15.103:0/2643496175 conn(0x55cbf1988020 msgr2=0x55cbf19a1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 -- 172.21.15.103:0/2643496175 shutdown_connections 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.376+0000 7f49b5a23740 1 -- 172.21.15.103:0/2643496175 wait complete. 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 -- start start 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 Processor -- start 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198a240 0x55cbf19abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf19ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf1996af0 0x55cbf19b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cbf19a8610 con 0x55cbf198ab60 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cbf19a7f40 con 0x55cbf1996af0 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cbf19a8ce0 con 0x55cbf198a240 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.377+0000 7f49b5a23740 10 client.0 _ll_register_callbacks cb 0x7f499c0e99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf19ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198a240 0x55cbf19abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf19ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35952/0 (socket says 172.21.15.103:35952) 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49a3e00640 1 -- 172.21.15.103:0/21027413 learned_addr learned my addr 172.21.15.103:0/21027413 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49b0e00640 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf1996af0 0x55cbf19b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49abe00640 1 -- 172.21.15.103:0/21027413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf1996af0 msgr2=0x55cbf19b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49abe00640 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf1996af0 0x55cbf19b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49abe00640 1 -- 172.21.15.103:0/21027413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 msgr2=0x55cbf19ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49abe00640 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf19ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49abe00640 1 -- 172.21.15.103:0/21027413 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf19b0f10 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49a3e00640 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf19ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f49abe00640 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198a240 0x55cbf19abb10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f499c0a83c0 tx=0x7f499c0e9ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c45a2612638d9119 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f499a000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.378+0000 7f499a000640 1 -- 172.21.15.103:0/21027413 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c097060 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4994004770 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f499a000640 1 -- 172.21.15.103:0/21027413 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f499c0e3390 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f499a000640 1 -- 172.21.15.103:0/21027413 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f499c0f5070 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 10 client.25135 fetch_fsmap learned FSMap version 26 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbf1a144e0 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f499a000640 1 -- 172.21.15.103:0/21027413 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4994004770 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f499a000640 1 -- 172.21.15.103:0/21027413 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f499c0b4ef0 con 0x55cbf198a240 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 10 client.25135 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:22:50.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 10 client.25135 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:22:50.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 20 client.25135 populate_metadata read hostname 'smithi103' 2024-09-11T00:22:50.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf1a18de0 0x55cbf1a17af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:50.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 4 client.25135 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:22:50.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.379+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 --> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] -- command(tid 0: {"prefix": "status"}) -- 0x55cbf18ff750 con 0x55cbf1a18de0 2024-09-11T00:22:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.380+0000 7f49a3e00640 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf1a18de0 0x55cbf1a17af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.380+0000 7f49a3e00640 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf1a18de0 0x55cbf1a17af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7af312dfed22f06 server_cookie=9b4f62d604887eb6 in_seq=0 out_seq=0 2024-09-11T00:22:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.380+0000 7f499a000640 10 client.25135 ms_handle_connect on v2:172.21.15.183:6836/335217865 2024-09-11T00:22:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f499a000640 1 -- 172.21.15.103:0/21027413 <== mds.2 v2:172.21.15.183:6836/335217865 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cbf18ff750 con 0x55cbf1a18de0 2024-09-11T00:22:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f499a000640 10 client.25135 handle_command_reply: tid=0 2024-09-11T00:22:50.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 client.25135 shutdown 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198a240 msgr2=0x55cbf19abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198a240 0x55cbf19abb10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f499c0a83c0 tx=0x7f499c0e9ed0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 shutdown_connections 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x55cbf1a18de0 0x55cbf1a17af0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cbf1996af0 0x55cbf19b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cbf198ab60 0x55cbf19ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f499a000640 0 client.25135 ms_handle_reset on v2:172.21.15.183:6836/335217865 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 --2- 172.21.15.103:0/21027413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cbf198a240 0x55cbf19abb10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 >> 172.21.15.103:0/21027413 conn(0x55cbf1988020 msgr2=0x55cbf1995870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 shutdown_connections 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 1 -- 172.21.15.103:0/21027413 wait complete. 2024-09-11T00:22:50.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.381+0000 7f49b5a23740 20 client.25135 trim_cache size 0 max 16384 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24628, 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 87.260705580999996, 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 173.04868290900001 2024-09-11T00:22:50.387 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:22:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49900b5370 msgr2=0x7f49900b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49900b5370 0x7f49900b7830 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f49a4005fd0 tx=0x7f49a4009500 comp rx=0 tx=0).stop 2024-09-11T00:22:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac104f90 msgr2=0x7f49ac0776b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac104f90 0x7f49ac0776b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d950 tx=0x7f499c00de10 comp rx=0 tx=0).stop 2024-09-11T00:22:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 shutdown_connections 2024-09-11T00:22:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f49900b5370 0x7f49900b7830 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49ac10b320 0x7f49ac078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f49ac10a9a0 0x7f49ac077bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 --2- 172.21.15.103:0/2528778443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49ac104f90 0x7f49ac0776b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:50.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 >> 172.21.15.103:0/2528778443 conn(0x7f49ac100a00 msgr2=0x7f49ac102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:50.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 shutdown_connections 2024-09-11T00:22:50.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:50.382+0000 7f49b2e00640 1 -- 172.21.15.103:0/2528778443 wait complete. 2024-09-11T00:22:50.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:50 smithi103 ceph-mon[30578]: pgmap v391: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:50 smithi183 ceph-mon[35934]: pgmap v391: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:50 smithi117 ceph-mon[35525]: pgmap v391: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:51.340 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:22:51.710 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:52.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.385+0000 7fdb03a00640 1 -- 172.21.15.103:0/3518393971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 msgr2=0x7fdafc111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.385+0000 7fdb03a00640 1 --2- 172.21.15.103:0/3518393971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 0x7fdafc111b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009ca0 tx=0x7fdaf402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.385+0000 7fdb03a00640 1 -- 172.21.15.103:0/3518393971 shutdown_connections 2024-09-11T00:22:52.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.385+0000 7fdb03a00640 1 --2- 172.21.15.103:0/3518393971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 0x7fdafc111b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.385+0000 7fdb03a00640 1 --2- 172.21.15.103:0/3518393971 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdafc10a9d0 0x7fdafc10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.385+0000 7fdb03a00640 1 --2- 172.21.15.103:0/3518393971 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 0x7fdafc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.385+0000 7fdb03a00640 1 -- 172.21.15.103:0/3518393971 >> 172.21.15.103:0/3518393971 conn(0x7fdafc1009f0 msgr2=0x7fdafc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.386+0000 7fdb03a00640 1 -- 172.21.15.103:0/3518393971 shutdown_connections 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.386+0000 7fdb03a00640 1 -- 172.21.15.103:0/3518393971 wait complete. 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 Processor -- start 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 -- start start 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 0x7fdafc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdafc10a9d0 0x7fdafc19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 0x7fdafc1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc114fd0 con 0x7fdafc10b350 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc114e50 con 0x7fdafc10a9d0 2024-09-11T00:22:52.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb03a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc115150 con 0x7fdafc104fc0 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.387+0000 7fdb01000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 0x7fdafc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 0x7fdafc19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59266/0 (socket says 172.21.15.103:59266) 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 -- 172.21.15.103:0/1669196390 learned_addr learned my addr 172.21.15.103:0/1669196390 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01a00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 0x7fdafc1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdafbe00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdafc10a9d0 0x7fdafc19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 -- 172.21.15.103:0/1669196390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdafc10a9d0 msgr2=0x7fdafc19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdafc10a9d0 0x7fdafc19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 -- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 msgr2=0x7fdafc1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 0x7fdafc1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 -- 172.21.15.103:0/1669196390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc1a14f0 con 0x7fdafc104fc0 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.388+0000 7fdb01000640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 0x7fdafc19b440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e9f0 tx=0x7fdaf000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.389+0000 7fdaf9600640 1 -- 172.21.15.103:0/1669196390 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000cd80 con 0x7fdafc104fc0 2024-09-11T00:22:52.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.389+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1a17e0 con 0x7fdafc104fc0 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.389+0000 7fdaf9600640 1 -- 172.21.15.103:0/1669196390 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf00054d0 con 0x7fdafc104fc0 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.389+0000 7fdaf9600640 1 -- 172.21.15.103:0/1669196390 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0010690 con 0x7fdafc104fc0 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.389+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1a7ea0 con 0x7fdafc104fc0 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.392+0000 7fdb065cd740 1 Processor -- start 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.392+0000 7fdb065cd740 1 -- start start 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681823b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x5568182475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818249ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdb065cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5568181b2d50 con 0x55681823bb60 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdb065cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5568181a2180 con 0x556818247af0 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdb065cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5568181a5f90 con 0x55681823b240 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdaf9600640 1 -- 172.21.15.103:0/1669196390 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fdaf0004e90 con 0x7fdafc104fc0 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818249ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818249ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33796/0 (socket says 172.21.15.103:33796) 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdafbe00640 1 -- 172.21.15.103:0/897152290 learned_addr learned my addr 172.21.15.103:0/897152290 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.393+0000 7fdb01000640 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x5568182475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdb01a00640 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681823b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdafbe00640 1 -- 172.21.15.103:0/897152290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 msgr2=0x55681823b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdafbe00640 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681823b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdafbe00640 1 -- 172.21.15.103:0/897152290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 msgr2=0x5568182475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdafbe00640 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x5568182475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdafbe00640 1 -- 172.21.15.103:0/897152290 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55681816ca70 con 0x556818247af0 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdafbe00640 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818249ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009c90 tx=0x7fdae8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f601f9ce1f72e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.394+0000 7fdaee000640 1 -- 172.21.15.103:0/897152290 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae801e070 con 0x556818247af0 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdaee000640 1 -- 172.21.15.103:0/897152290 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdae8019ae0 con 0x556818247af0 2024-09-11T00:22:52.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdaee000640 1 -- 172.21.15.103:0/897152290 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae800e940 con 0x556818247af0 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 -- 172.21.15.103:0/897152290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 msgr2=0x556818249ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818249ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009c90 tx=0x7fdae8011810 comp rx=0 tx=0).stop 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 -- 172.21.15.103:0/897152290 shutdown_connections 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818249ee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x5568182475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 --2- 172.21.15.103:0/897152290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681823b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 -- 172.21.15.103:0/897152290 >> 172.21.15.103:0/897152290 conn(0x556818239020 msgr2=0x55681823a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.395+0000 7fdb065cd740 1 -- 172.21.15.103:0/897152290 shutdown_connections 2024-09-11T00:22:52.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.396+0000 7fdb065cd740 1 -- 172.21.15.103:0/897152290 wait complete. 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.396+0000 7fdaf9600640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdae40b5280 0x7fdae40b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.396+0000 7fdaf9600640 1 -- 172.21.15.103:0/1669196390 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdaf00e6530 con 0x7fdafc104fc0 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.396+0000 7fdafbe00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdae40b5280 0x7fdae40b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.396+0000 7fdb065cd740 1 -- start start 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.396+0000 7fdb065cd740 1 Processor -- start 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x556818247590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681823bb60 0x556818244a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556818247af0 0x556818245190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb065cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5568180035f0 con 0x556818247af0 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb065cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55681813d460 con 0x55681823bb60 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb065cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55681824a840 con 0x55681823b240 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb065cd740 10 client.0 _ll_register_callbacks cb 0x55681814d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdafbe00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdae40b5280 0x7fdae40b7740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009a50 tx=0x7fdae8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:52.458 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb01000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x556818247590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681823bb60 0x556818244a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb01000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x556818247590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59278/0 (socket says 172.21.15.103:59278) 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb01000640 1 -- 172.21.15.103:0/1309599859 learned_addr learned my addr 172.21.15.103:0/1309599859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681823bb60 0x556818244a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33804/0 (socket says 172.21.15.103:33804) 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.397+0000 7fdb01a00640 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556818247af0 0x556818245190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb01000640 1 -- 172.21.15.103:0/1309599859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681823bb60 msgr2=0x556818244a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb01000640 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681823bb60 0x556818244a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb01000640 1 -- 172.21.15.103:0/1309599859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556818247af0 msgr2=0x556818245190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb01000640 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556818247af0 0x556818245190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb01000640 1 -- 172.21.15.103:0/1309599859 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556817f00540 con 0x55681823b240 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb01a00640 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556818247af0 0x556818245190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb01000640 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x556818247590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00097d0 tx=0x7fdaf00e9f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=319fd54022541ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:52.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdae2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdae2a00640 1 -- 172.21.15.103:0/1309599859 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00ac2b0 con 0x55681823b240 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55681813fa70 con 0x55681823b240 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.398+0000 7fdae2a00640 1 -- 172.21.15.103:0/1309599859 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdaf00e3d40 con 0x55681823b240 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdae2a00640 1 -- 172.21.15.103:0/1309599859 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdaf0005680 con 0x55681823b240 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 10 client.25147 fetch_fsmap learned FSMap version 26 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55681815a6f0 con 0x55681823b240 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdae2a00640 1 -- 172.21.15.103:0/1309599859 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55681813fa70 con 0x55681823b240 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdae2a00640 1 -- 172.21.15.103:0/1309599859 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdaf00b58a0 con 0x55681823b240 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 10 client.25147 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 10 client.25147 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 20 client.25147 populate_metadata read hostname 'smithi103' 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x556818258800 0x556818259c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 4 client.25147 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:52.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.399+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 --> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568181b4ca0 con 0x556818258800 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.400+0000 7fdafbe00640 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x556818258800 0x556818259c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.400+0000 7fdafbe00640 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x556818258800 0x556818259c70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9efb47a88a50a079 server_cookie=747be59c1250f2d2 in_seq=0 out_seq=0 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.400+0000 7fdae2a00640 10 client.25147 ms_handle_connect on v2:172.21.15.183:6838/424027451 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.401+0000 7fdae2a00640 1 -- 172.21.15.103:0/1309599859 <== mds.1 v2:172.21.15.183:6838/424027451 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5568181b4ca0 con 0x556818258800 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.401+0000 7fdae2a00640 10 client.25147 handle_command_reply: tid=0 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.401+0000 7fdb065cd740 1 client.25147 shutdown 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 msgr2=0x556818247590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x556818247590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00097d0 tx=0x7fdaf00e9f90 comp rx=0 tx=0).stop 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 shutdown_connections 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x556818258800 0x556818259c70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556818247af0 0x556818245190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55681823bb60 0x556818244a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.461 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 --2- 172.21.15.103:0/1309599859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x556818247590 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdae2a00640 0 client.25147 ms_handle_reset on v2:172.21.15.183:6838/424027451 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 >> 172.21.15.103:0/1309599859 conn(0x556818239020 msgr2=0x55681823ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 shutdown_connections 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 1 -- 172.21.15.103:0/1309599859 wait complete. 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.402+0000 7fdb065cd740 20 client.25147 trim_cache size 0 max 16384 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 Processor -- start 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 -- start start 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823b240 0x556818259c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823bb60 0x55681823f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818239d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556818257fb0 con 0x55681823b240 2024-09-11T00:22:52.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556818258680 con 0x556818247af0 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb065cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556818258d50 con 0x55681823bb60 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb01000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823b240 0x556818259c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb01000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823b240 0x556818259c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54570/0 (socket says 172.21.15.103:54570) 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb01000640 1 -- 172.21.15.103:0/569997306 learned_addr learned my addr 172.21.15.103:0/569997306 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdb01a00640 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818239d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.421+0000 7fdafbe00640 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823bb60 0x55681823f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdb01000640 1 -- 172.21.15.103:0/569997306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823bb60 msgr2=0x55681823f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdb01000640 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823bb60 0x55681823f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdb01000640 1 -- 172.21.15.103:0/569997306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 msgr2=0x556818239d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdb01000640 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818239d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdb01000640 1 -- 172.21.15.103:0/569997306 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556818244d70 con 0x55681823b240 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdb01000640 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823b240 0x556818259c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00b13d0 tx=0x7fdaf00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86db93b9d812f5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:52.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdaed600640 1 -- 172.21.15.103:0/569997306 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00ac0f0 con 0x55681823b240 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdaed600640 1 -- 172.21.15.103:0/569997306 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf00cf040 con 0x55681823b240 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.422+0000 7fdaed600640 1 -- 172.21.15.103:0/569997306 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00acea0 con 0x55681823b240 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 -- 172.21.15.103:0/569997306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823b240 msgr2=0x556818259c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823b240 0x556818259c70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00b13d0 tx=0x7fdaf00048f0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 -- 172.21.15.103:0/569997306 shutdown_connections 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x556818239d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823bb60 0x55681823f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 --2- 172.21.15.103:0/569997306 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823b240 0x556818259c70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 -- 172.21.15.103:0/569997306 >> 172.21.15.103:0/569997306 conn(0x556818239020 msgr2=0x55681823fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 -- 172.21.15.103:0/569997306 shutdown_connections 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 -- 172.21.15.103:0/569997306 wait complete. 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 -- start start 2024-09-11T00:22:52.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.423+0000 7fdb065cd740 1 Processor -- start 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681825ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x55681825d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb065cd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x55681825d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb065cd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556818259af0 con 0x55681823bb60 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb065cd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x556818259420 con 0x556818247af0 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb065cd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55681825a1c0 con 0x55681823b240 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb065cd740 10 client.0 _ll_register_callbacks cb 0x7fdaf00a8640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x55681825d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdafbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x55681825d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54582/0 (socket says 172.21.15.103:54582) 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdafbe00640 1 -- 172.21.15.103:0/3733985303 learned_addr learned my addr 172.21.15.103:0/3733985303 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb01000640 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681825ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.424+0000 7fdb01a00640 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x55681825d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdafbe00640 1 -- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 msgr2=0x55681825ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdafbe00640 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681825ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdafbe00640 1 -- 172.21.15.103:0/3733985303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 msgr2=0x55681825d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdafbe00640 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x55681825d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdafbe00640 1 -- 172.21.15.103:0/3733985303 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556818262140 con 0x55681823bb60 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdafbe00640 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x55681825d370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdae80098c0 tx=0x7fdae8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4e94cbcede25ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdae2a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdae2a00640 1 -- 172.21.15.103:0/3733985303 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae801e040 con 0x55681823bb60 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf00cf040 con 0x55681823bb60 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.425+0000 7fdae2a00640 1 -- 172.21.15.103:0/3733985303 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdae8004ad0 con 0x55681823bb60 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdae2a00640 1 -- 172.21.15.103:0/3733985303 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdae8004d10 con 0x55681823bb60 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 10 client.15117 fetch_fsmap learned FSMap version 26 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5568182c5700 con 0x55681823bb60 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdae2a00640 1 -- 172.21.15.103:0/3733985303 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaf00cf040 con 0x55681823bb60 2024-09-11T00:22:52.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdae2a00640 1 -- 172.21.15.103:0/3733985303 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdaf00051b0 con 0x55681823bb60 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 10 client.15117 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 10 client.15117 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 20 client.15117 populate_metadata read hostname 'smithi103' 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x5568182ca050 0x5568182c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 4 client.15117 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 --> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] -- command(tid 0: {"prefix": "status"}) -- 0x5568181b0750 con 0x5568182ca050 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.426+0000 7fdb01000640 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x5568182ca050 0x5568182c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb01000640 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x5568182ca050 0x5568182c8d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7ac1120c483f232 server_cookie=a79db3fb26b230ad in_seq=0 out_seq=0 2024-09-11T00:22:52.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdae2a00640 10 client.15117 ms_handle_connect on v2:172.21.15.183:6838/424027451 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdae2a00640 1 -- 172.21.15.103:0/3733985303 <== mds.1 v2:172.21.15.183:6838/424027451 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5568181b0750 con 0x5568182ca050 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdae2a00640 10 client.15117 handle_command_reply: tid=0 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 client.15117 shutdown 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 msgr2=0x55681825d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x55681825d370 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdae80098c0 tx=0x7fdae8009850 comp rx=0 tx=0).stop 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 shutdown_connections 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x5568182ca050 0x5568182c8d60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x556818247af0 0x55681825d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55681823bb60 0x55681825d370 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdae2a00640 0 client.15117 ms_handle_reset on v2:172.21.15.183:6838/424027451 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 --2- 172.21.15.103:0/3733985303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55681823b240 0x55681825ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.468 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.427+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 >> 172.21.15.103:0/3733985303 conn(0x556818239020 msgr2=0x556818239d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 shutdown_connections 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb065cd740 1 -- 172.21.15.103:0/3733985303 wait complete. 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24643, 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:22:52.469 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 90.321288007999996, 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 169.75647907999999 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb065cd740 20 client.15117 trim_cache size 0 max 16384 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdae40b5280 msgr2=0x7fdae40b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdae40b5280 0x7fdae40b7740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009a50 tx=0x7fdae8000bc0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 msgr2=0x7fdafc19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:52.470 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 0x7fdafc19b440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e9f0 tx=0x7fdaf000edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 shutdown_connections 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdae40b5280 0x7fdae40b7740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdafc10b350 0x7fdafc1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdafc10a9d0 0x7fdafc19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 --2- 172.21.15.103:0/1669196390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdafc104fc0 0x7fdafc19b440 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 >> 172.21.15.103:0/1669196390 conn(0x7fdafc1009f0 msgr2=0x7fdafc102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 shutdown_connections 2024-09-11T00:22:52.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:52.428+0000 7fdb03a00640 1 -- 172.21.15.103:0/1669196390 wait complete. 2024-09-11T00:22:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:52 smithi103 ceph-mon[30578]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:52 smithi183 ceph-mon[35934]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:52.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:52 smithi117 ceph-mon[35525]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:53.313 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:22:53.686 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.301+0000 7fa36f200640 1 -- 172.21.15.103:0/3218769481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368102dc0 msgr2=0x7fa3681031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.301+0000 7fa36f200640 1 --2- 172.21.15.103:0/3218769481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368102dc0 0x7fa3681031a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa358009900 tx=0x7fa35802a410 comp rx=0 tx=0).stop 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.301+0000 7fa36f200640 1 -- 172.21.15.103:0/3218769481 shutdown_connections 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.301+0000 7fa36f200640 1 --2- 172.21.15.103:0/3218769481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa368109150 0x7fa36810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.301+0000 7fa36f200640 1 --2- 172.21.15.103:0/3218769481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3681087d0 0x7fa368108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.301+0000 7fa36f200640 1 --2- 172.21.15.103:0/3218769481 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368102dc0 0x7fa3681031a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.301+0000 7fa36f200640 1 -- 172.21.15.103:0/3218769481 >> 172.21.15.103:0/3218769481 conn(0x7fa3680fe830 msgr2=0x7fa368100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.302+0000 7fa36f200640 1 -- 172.21.15.103:0/3218769481 shutdown_connections 2024-09-11T00:22:54.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.302+0000 7fa36f200640 1 -- 172.21.15.103:0/3218769481 wait complete. 2024-09-11T00:22:54.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.302+0000 7fa36f200640 1 Processor -- start 2024-09-11T00:22:54.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36f200640 1 -- start start 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36f200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa368102dc0 0x7fa36819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36f200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3681087d0 0x7fa36819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368109150 0x7fa3681a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa368112e20 con 0x7fa368109150 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36f200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa368112ca0 con 0x7fa3681087d0 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36f200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa368112fa0 con 0x7fa368102dc0 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa367e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa368102dc0 0x7fa36819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa367e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa368102dc0 0x7fa36819b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59300/0 (socket says 172.21.15.103:59300) 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa36d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368109150 0x7fa3681a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa367e00640 1 -- 172.21.15.103:0/1064569751 learned_addr learned my addr 172.21.15.103:0/1064569751 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.303+0000 7fa367400640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3681087d0 0x7fa36819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa367e00640 1 -- 172.21.15.103:0/1064569751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3681087d0 msgr2=0x7fa36819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa367e00640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3681087d0 0x7fa36819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa367e00640 1 -- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368109150 msgr2=0x7fa3681a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa367e00640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368109150 0x7fa3681a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa367e00640 1 -- 172.21.15.103:0/1064569751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3681a1540 con 0x7fa368102dc0 2024-09-11T00:22:54.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa367e00640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa368102dc0 0x7fa36819b490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa358005ab0 tx=0x7fa358003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:54.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa364c00640 1 -- 172.21.15.103:0/1064569751 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35803f070 con 0x7fa368102dc0 2024-09-11T00:22:54.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3681a17d0 con 0x7fa368102dc0 2024-09-11T00:22:54.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa364c00640 1 -- 172.21.15.103:0/1064569751 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa358031b60 con 0x7fa368102dc0 2024-09-11T00:22:54.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.304+0000 7fa364c00640 1 -- 172.21.15.103:0/1064569751 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa358035030 con 0x7fa368102dc0 2024-09-11T00:22:54.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.305+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3681a7ef0 con 0x7fa368102dc0 2024-09-11T00:22:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.307+0000 7fa371df0740 1 Processor -- start 2024-09-11T00:22:54.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa371df0740 1 -- start start 2024-09-11T00:22:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa371df0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1240 0x563bfa1f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa371df0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa371df0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1ffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa371df0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bfa168d50 con 0x563bfa1f1b60 2024-09-11T00:22:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa371df0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563bfa158180 con 0x563bfa1f1240 2024-09-11T00:22:54.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa371df0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bfa15bf90 con 0x563bfa1fdaf0 2024-09-11T00:22:54.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa36d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1240 0x563bfa1f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa367400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1ffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa36d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1240 0x563bfa1f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33832/0 (socket says 172.21.15.103:33832) 2024-09-11T00:22:54.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa36d200640 1 -- 172.21.15.103:0/3263406692 learned_addr learned my addr 172.21.15.103:0/3263406692 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:54.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa367400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1ffee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59312/0 (socket says 172.21.15.103:59312) 2024-09-11T00:22:54.311 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.308+0000 7fa364c00640 1 -- 172.21.15.103:0/1064569751 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fa35803b460 con 0x7fa368102dc0 2024-09-11T00:22:54.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa36d200640 1 -- 172.21.15.103:0/3263406692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 msgr2=0x563bfa1ffee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa36d200640 1 --2- 172.21.15.103:0/3263406692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1ffee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa36d200640 1 -- 172.21.15.103:0/3263406692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 msgr2=0x563bfa1fd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa36d200640 1 --2- 172.21.15.103:0/3263406692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa36d200640 1 -- 172.21.15.103:0/3263406692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bfa122a70 con 0x563bfa1f1240 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa36d200640 1 --2- 172.21.15.103:0/3263406692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1240 0x563bfa1f1620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fd0 tx=0x7fa3600118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7986c5959812629d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa35d600640 1 -- 172.21.15.103:0/3263406692 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360004d20 con 0x563bfa1f1240 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa35d600640 1 -- 172.21.15.103:0/3263406692 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa360004050 con 0x563bfa1f1240 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.309+0000 7fa35d600640 1 -- 172.21.15.103:0/3263406692 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360005260 con 0x563bfa1f1240 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa371df0740 1 -- 172.21.15.103:0/3263406692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1240 msgr2=0x563bfa1f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa371df0740 1 --2- 172.21.15.103:0/3263406692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1240 0x563bfa1f1620 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa360009fd0 tx=0x7fa3600118d0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa364c00640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa34c0b5370 0x7fa34c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa371df0740 1 -- 172.21.15.103:0/3263406692 shutdown_connections 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa371df0740 1 --2- 172.21.15.103:0/3263406692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa371df0740 1 --2- 172.21.15.103:0/3263406692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa371df0740 1 --2- 172.21.15.103:0/3263406692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1240 0x563bfa1f1620 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa371df0740 1 -- 172.21.15.103:0/3263406692 >> 172.21.15.103:0/3263406692 conn(0x563bfa1ef020 msgr2=0x563bfa1f0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:54.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa364c00640 1 -- 172.21.15.103:0/1064569751 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa358106b20 con 0x7fa368102dc0 2024-09-11T00:22:54.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.310+0000 7fa367e00640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa34c0b5370 0x7fa34c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.311+0000 7fa371df0740 1 -- 172.21.15.103:0/3263406692 shutdown_connections 2024-09-11T00:22:54.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.311+0000 7fa371df0740 1 -- 172.21.15.103:0/3263406692 wait complete. 2024-09-11T00:22:54.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.311+0000 7fa371df0740 1 -- start start 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.311+0000 7fa367e00640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa34c0b5370 0x7fa34c0b7830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa3580ccdd0 tx=0x7fa358009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.311+0000 7fa371df0740 1 Processor -- start 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa371df0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa1faa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa371df0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1b60 0x563bfa1faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa371df0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa1fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa371df0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bf9fb95f0 con 0x563bfa1f1240 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa371df0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563bfa0f3460 con 0x563bfa1fdaf0 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa371df0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bfa200840 con 0x563bfa1f1b60 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa371df0740 10 client.0 _ll_register_callbacks cb 0x563bfa103210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa36d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa1faa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa36d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa1faa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54628/0 (socket says 172.21.15.103:54628) 2024-09-11T00:22:54.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa36d200640 1 -- 172.21.15.103:0/1946650056 learned_addr learned my addr 172.21.15.103:0/1946650056 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa367400640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1b60 0x563bfa1faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.312+0000 7fa36d200640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa1fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa367400640 1 -- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 msgr2=0x563bfa1fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa367400640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa1fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa367400640 1 -- 172.21.15.103:0/1946650056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 msgr2=0x563bfa1faa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa367400640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa1faa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa367400640 1 -- 172.21.15.103:0/1946650056 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf9eb6540 con 0x563bfa1f1b60 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa36d200640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa1faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa367400640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1b60 0x563bfa1faf60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa368107ea0 tx=0x7fa354012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4db274bcc77b49d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa352000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa352000640 1 -- 172.21.15.103:0/1946650056 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35400f920 con 0x563bfa1f1b60 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563bfa0f5a70 con 0x563bfa1f1b60 2024-09-11T00:22:54.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa352000640 1 -- 172.21.15.103:0/1946650056 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa35400fac0 con 0x563bfa1f1b60 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.313+0000 7fa352000640 1 -- 172.21.15.103:0/1946650056 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa35400fcc0 con 0x563bfa1f1b60 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 10 client.25165 fetch_fsmap learned FSMap version 26 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563bfa1106f0 con 0x563bfa1f1b60 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa352000640 1 -- 172.21.15.103:0/1946650056 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563bfa0f5a70 con 0x563bfa1f1b60 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa352000640 1 -- 172.21.15.103:0/1946650056 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa35401c420 con 0x563bfa1f1b60 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 10 client.25165 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 10 client.25165 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 20 client.25165 populate_metadata read hostname 'smithi103' 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa20e870 0x563bfa20fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 4 client.25165 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.314+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bfa16aca0 con 0x563bfa20e870 2024-09-11T00:22:54.319 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.315+0000 7fa36d200640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa20e870 0x563bfa20fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.315+0000 7fa36d200640 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa20e870 0x563bfa20fce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b9cc01b31a10c4 server_cookie=c9c8bd598ceee7d7 in_seq=0 out_seq=0 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.315+0000 7fa352000640 10 client.25165 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa352000640 1 -- 172.21.15.103:0/1946650056 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563bfa16aca0 con 0x563bfa20e870 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa352000640 10 client.25165 handle_command_reply: tid=0 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 client.25165 shutdown 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1b60 msgr2=0x563bfa1faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1b60 0x563bfa1faf60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa368107ea0 tx=0x7fa354012b60 comp rx=0 tx=0).stop 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 shutdown_connections 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa1fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa20e870 0x563bfa20fce0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1b60 0x563bfa1faf60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 --2- 172.21.15.103:0/1946650056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa1faa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa352000640 0 client.25165 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:22:54.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.316+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 >> 172.21.15.103:0/1946650056 conn(0x563bfa1ef020 msgr2=0x563bfa1f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:54.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.317+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 shutdown_connections 2024-09-11T00:22:54.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.317+0000 7fa371df0740 1 -- 172.21.15.103:0/1946650056 wait complete. 2024-09-11T00:22:54.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.317+0000 7fa371df0740 20 client.25165 trim_cache size 0 max 16384 2024-09-11T00:22:54.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 Processor -- start 2024-09-11T00:22:54.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 -- start start 2024-09-11T00:22:54.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1240 0x563bfa20fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa20d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bfa20e020 con 0x563bfa1f1b60 2024-09-11T00:22:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563bfa20e6f0 con 0x563bfa1fdaf0 2024-09-11T00:22:54.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.335+0000 7fa371df0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bfa20edc0 con 0x563bfa1f1240 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1f5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54630/0 (socket says 172.21.15.103:54630) 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 -- 172.21.15.103:0/496557370 learned_addr learned my addr 172.21.15.103:0/496557370 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa36d200640 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1240 0x563bfa20fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa36d200640 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa20d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 -- 172.21.15.103:0/496557370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1240 msgr2=0x563bfa20fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1240 0x563bfa20fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 -- 172.21.15.103:0/496557370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 msgr2=0x563bfa20d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa20d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 -- 172.21.15.103:0/496557370 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bfa1eee50 con 0x563bfa1f1b60 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa36d200640 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa20d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.336+0000 7fa367400640 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1f5400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa354001eb0 tx=0x7fa354005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca62dbcf173adeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa35cc00640 1 -- 172.21.15.103:0/496557370 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35400f920 con 0x563bfa1f1b60 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa35cc00640 1 -- 172.21.15.103:0/496557370 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa35400fac0 con 0x563bfa1f1b60 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa35cc00640 1 -- 172.21.15.103:0/496557370 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3540035f0 con 0x563bfa1f1b60 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 -- 172.21.15.103:0/496557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 msgr2=0x563bfa1f5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1f5400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa354001eb0 tx=0x7fa354005880 comp rx=0 tx=0).stop 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 -- 172.21.15.103:0/496557370 shutdown_connections 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1fdaf0 0x563bfa20d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1b60 0x563bfa1f5400 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 --2- 172.21.15.103:0/496557370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1f1240 0x563bfa20fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 -- 172.21.15.103:0/496557370 >> 172.21.15.103:0/496557370 conn(0x563bfa1ef020 msgr2=0x563bfa1f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:54.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 -- 172.21.15.103:0/496557370 shutdown_connections 2024-09-11T00:22:54.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.337+0000 7fa371df0740 1 -- 172.21.15.103:0/496557370 wait complete. 2024-09-11T00:22:54.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.338+0000 7fa371df0740 1 -- start start 2024-09-11T00:22:54.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.338+0000 7fa371df0740 1 Processor -- start 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.338+0000 7fa371df0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa218840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa371df0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1b60 0x563bfa1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa371df0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa371df0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563bfa20fb60 con 0x563bfa1f1240 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa371df0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563bfa20f490 con 0x563bfa1f1b60 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa371df0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563bfa210230 con 0x563bfa1fdaf0 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa371df0740 10 client.0 _ll_register_callbacks cb 0x7fa354009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa36d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa218840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1b60 0x563bfa1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1b60 0x563bfa1f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33866/0 (socket says 172.21.15.103:33866) 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 -- 172.21.15.103:0/75052260 learned_addr learned my addr 172.21.15.103:0/75052260 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa36d200640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa218840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54646/0 (socket says 172.21.15.103:54646) 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa36d200640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 -- 172.21.15.103:0/75052260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 msgr2=0x563bfa1f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 -- 172.21.15.103:0/75052260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 msgr2=0x563bfa218840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa218840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa367400640 1 -- 172.21.15.103:0/75052260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bfa2174a0 con 0x563bfa1f1b60 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.339+0000 7fa36d200640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa367400640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1b60 0x563bfa1f7a60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa354000c00 tx=0x7fa3540052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791db1a9b4811d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa352000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:22:54.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa352000640 1 -- 172.21.15.103:0/75052260 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35401cad0 con 0x563bfa1f1b60 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa35400fac0 con 0x563bfa1f1b60 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa352000640 1 -- 172.21.15.103:0/75052260 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa354018070 con 0x563bfa1f1b60 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa352000640 1 -- 172.21.15.103:0/75052260 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa35403c050 con 0x563bfa1f1b60 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa371df0740 10 client.25142 fetch_fsmap learned FSMap version 26 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.340+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563bfa27b330 con 0x563bfa1f1b60 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa352000640 1 -- 172.21.15.103:0/75052260 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa35400fac0 con 0x563bfa1f1b60 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa352000640 1 -- 172.21.15.103:0/75052260 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa3540364e0 con 0x563bfa1f1b60 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa371df0740 10 client.25142 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa371df0740 10 client.25142 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa371df0740 20 client.25142 populate_metadata read hostname 'smithi103' 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa371df0740 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa27fc30 0x563bfa27e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa371df0740 4 client.25142 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:22:54.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "status"}) -- 0x563bfa166750 con 0x563bfa27fc30 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.341+0000 7fa36d200640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa27fc30 0x563bfa27e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa36d200640 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa27fc30 0x563bfa27e940 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c86a9252f7b2230f server_cookie=11699e9c94b24240 in_seq=0 out_seq=0 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa352000640 10 client.25142 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa352000640 1 -- 172.21.15.103:0/75052260 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563bfa166750 con 0x563bfa27fc30 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa352000640 10 client.25142 handle_command_reply: tid=0 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 client.25142 shutdown 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1b60 msgr2=0x563bfa1f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1b60 0x563bfa1f7a60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa354000c00 tx=0x7fa3540052e0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 shutdown_connections 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563bfa27fc30 0x563bfa27e940 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563bfa1fdaf0 0x563bfa1f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563bfa1f1b60 0x563bfa1f7a60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 --2- 172.21.15.103:0/75052260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563bfa1f1240 0x563bfa218840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa352000640 0 client.25142 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 >> 172.21.15.103:0/75052260 conn(0x563bfa1ef020 msgr2=0x563bfa1fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 shutdown_connections 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.342+0000 7fa371df0740 1 -- 172.21.15.103:0/75052260 wait complete. 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa371df0740 20 client.25142 trim_cache size 0 max 16384 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24656, 2024-09-11T00:22:54.349 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 145.28855692100001, 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 160.26463719700001 2024-09-11T00:22:54.350 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa34c0b5370 msgr2=0x7fa34c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa34c0b5370 0x7fa34c0b7830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa3580ccdd0 tx=0x7fa358009260 comp rx=0 tx=0).stop 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa368102dc0 msgr2=0x7fa36819b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa368102dc0 0x7fa36819b490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa358005ab0 tx=0x7fa358003c10 comp rx=0 tx=0).stop 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 shutdown_connections 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa34c0b5370 0x7fa34c0b7830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa368109150 0x7fa3681a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3681087d0 0x7fa36819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 --2- 172.21.15.103:0/1064569751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa368102dc0 0x7fa36819b490 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 >> 172.21.15.103:0/1064569751 conn(0x7fa3680fe830 msgr2=0x7fa368104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.343+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 shutdown_connections 2024-09-11T00:22:54.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:54.344+0000 7fa36f200640 1 -- 172.21.15.103:0/1064569751 wait complete. 2024-09-11T00:22:54.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:54 smithi183 ceph-mon[35934]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:54.865 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:54 smithi103 ceph-mon[30578]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:54 smithi117 ceph-mon[35525]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:22:55.297 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T00:22:55.297 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T00:22:55.307 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:22:55.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T00:22:55.678 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:56.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 -- 172.21.15.103:0/3239549353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618078b70 msgr2=0x7fc618078ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.320 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 --2- 172.21.15.103:0/3239549353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618078b70 0x7fc618078ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc604009900 tx=0x7fc60402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 -- 172.21.15.103:0/3239549353 shutdown_connections 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 --2- 172.21.15.103:0/3239549353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc618079530 0x7fc618079e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 --2- 172.21.15.103:0/3239549353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618078b70 0x7fc618078ff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 --2- 172.21.15.103:0/3239549353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc618077a10 0x7fc618077e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 -- 172.21.15.103:0/3239549353 >> 172.21.15.103:0/3239549353 conn(0x7fc618100550 msgr2=0x7fc618102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 -- 172.21.15.103:0/3239549353 shutdown_connections 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.306+0000 7fc61d800640 1 -- 172.21.15.103:0/3239549353 wait complete. 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 Processor -- start 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 -- start start 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618077a10 0x7fc61819f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc618078b70 0x7fc61819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc618079530 0x7fc6181a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc618114d20 con 0x7fc618077a10 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc618114ba0 con 0x7fc618079530 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.307+0000 7fc61d800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc618114ea0 con 0x7fc618078b70 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618077a10 0x7fc61819f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618077a10 0x7fc61819f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54654/0 (socket says 172.21.15.103:54654) 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 -- 172.21.15.103:0/3426168121 learned_addr learned my addr 172.21.15.103:0/3426168121 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616000640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc618078b70 0x7fc61819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc617400640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc618079530 0x7fc6181a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 -- 172.21.15.103:0/3426168121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc618078b70 msgr2=0x7fc61819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc618078b70 0x7fc61819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 -- 172.21.15.103:0/3426168121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc618079530 msgr2=0x7fc6181a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc618079530 0x7fc6181a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 -- 172.21.15.103:0/3426168121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6181a0f60 con 0x7fc618077a10 2024-09-11T00:22:56.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.308+0000 7fc616a00640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618077a10 0x7fc61819f430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9f0 tx=0x7fc60800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.309+0000 7fc60f400640 1 -- 172.21.15.103:0/3426168121 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60800cd80 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.309+0000 7fc60f400640 1 -- 172.21.15.103:0/3426168121 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6080054d0 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.309+0000 7fc60f400640 1 -- 172.21.15.103:0/3426168121 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608010690 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.309+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6181a1250 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.309+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6181a7930 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.310+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc618078ff0 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.312+0000 7fc60f400640 1 -- 172.21.15.103:0/3426168121 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7fc608004e90 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.314+0000 7fc60f400640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc5fc0b5370 0x7fc5fc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.314+0000 7fc60f400640 1 -- 172.21.15.103:0/3426168121 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc6080e6560 con 0x7fc618077a10 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.314+0000 7fc616000640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc5fc0b5370 0x7fc5fc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.315+0000 7fc616000640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc5fc0b5370 0x7fc5fc0b7830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc604031500 tx=0x7fc604005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:56.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.318+0000 7fc60f400640 1 -- 172.21.15.103:0/3426168121 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc6080aced0 con 0x7fc618077a10 2024-09-11T00:22:56.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.523+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fc618063590 con 0x7fc5fc0b5370 2024-09-11T00:22:56.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.524+0000 7fc60f400640 1 -- 172.21.15.103:0/3426168121 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fc618063590 con 0x7fc5fc0b5370 2024-09-11T00:22:56.526 INFO:teuthology.orchestra.run.smithi103.stdout:Backend: cephadm 2024-09-11T00:22:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:Available: Yes 2024-09-11T00:22:56.527 INFO:teuthology.orchestra.run.smithi103.stdout:Paused: No 2024-09-11T00:22:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc5fc0b5370 msgr2=0x7fc5fc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc5fc0b5370 0x7fc5fc0b7830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc604031500 tx=0x7fc604005a20 comp rx=0 tx=0).stop 2024-09-11T00:22:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618077a10 msgr2=0x7fc61819f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618077a10 0x7fc61819f430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc60800e9f0 tx=0x7fc60800edf0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 shutdown_connections 2024-09-11T00:22:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc5fc0b5370 0x7fc5fc0b7830 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc618079530 0x7fc6181a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc618078b70 0x7fc61819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 --2- 172.21.15.103:0/3426168121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc618077a10 0x7fc61819f430 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 >> 172.21.15.103:0/3426168121 conn(0x7fc618100550 msgr2=0x7fc618102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 shutdown_connections 2024-09-11T00:22:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:56.527+0000 7fc61d800640 1 -- 172.21.15.103:0/3426168121 wait complete. 2024-09-11T00:22:56.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:56 smithi183 ceph-mon[35934]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:56.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:56 smithi103 ceph-mon[30578]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:56 smithi117 ceph-mon[35525]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:22:57.364 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T00:22:57.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:57 smithi103 ceph-mon[30578]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:22:57.734 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:57.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:57 smithi183 ceph-mon[35934]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:22:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:57 smithi117 ceph-mon[35525]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:22:58.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 -- 172.21.15.103:0/2406101134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c076f00 msgr2=0x7f7c7c075560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:58.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 --2- 172.21.15.103:0/2406101134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c076f00 0x7f7c7c075560 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009900 tx=0x7f7c6c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 -- 172.21.15.103:0/2406101134 shutdown_connections 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 --2- 172.21.15.103:0/2406101134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c10a840 0x7f7c7c10cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 --2- 172.21.15.103:0/2406101134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c7c075aa0 0x7f7c7c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 --2- 172.21.15.103:0/2406101134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c076f00 0x7f7c7c075560 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 -- 172.21.15.103:0/2406101134 >> 172.21.15.103:0/2406101134 conn(0x7f7c7c0fe180 msgr2=0x7f7c7c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 -- 172.21.15.103:0/2406101134 shutdown_connections 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.364+0000 7f7c82400640 1 -- 172.21.15.103:0/2406101134 wait complete. 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.365+0000 7f7c82400640 1 Processor -- start 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.365+0000 7f7c82400640 1 -- start start 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.365+0000 7f7c82400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c7c075aa0 0x7f7c7c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c82400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c076f00 0x7f7c7c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c82400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c10a840 0x7f7c7c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c82400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c110700 con 0x7f7c7c076f00 2024-09-11T00:22:58.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c82400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c110580 con 0x7f7c7c10a840 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c82400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c110880 con 0x7f7c7c075aa0 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c73e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c076f00 0x7f7c7c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c81a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c10a840 0x7f7c7c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c81000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c7c075aa0 0x7f7c7c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c73e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c076f00 0x7f7c7c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54664/0 (socket says 172.21.15.103:54664) 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c73e00640 1 -- 172.21.15.103:0/72695757 learned_addr learned my addr 172.21.15.103:0/72695757 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c81a00640 1 -- 172.21.15.103:0/72695757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c7c075aa0 msgr2=0x7f7c7c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c81a00640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c7c075aa0 0x7f7c7c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c81a00640 1 -- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c076f00 msgr2=0x7f7c7c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c81a00640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c076f00 0x7f7c7c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c81a00640 1 -- 172.21.15.103:0/72695757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c19c890 con 0x7f7c7c10a840 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.366+0000 7f7c73e00640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c076f00 0x7f7c7c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.367+0000 7f7c81a00640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c10a840 0x7f7c7c19bff0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c74005fd0 tx=0x7f7c7400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.367+0000 7f7c7a000640 1 -- 172.21.15.103:0/72695757 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400cdf0 con 0x7f7c7c10a840 2024-09-11T00:22:58.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.367+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7c19cb80 con 0x7f7c7c10a840 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.367+0000 7f7c7a000640 1 -- 172.21.15.103:0/72695757 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c74005020 con 0x7f7c7c10a840 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.367+0000 7f7c7a000640 1 -- 172.21.15.103:0/72695757 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74010690 con 0x7f7c7c10a840 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.367+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7c1a3260 con 0x7f7c7c10a840 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.368+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7c109650 con 0x7f7c7c10a840 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.370+0000 7f7c7a000640 1 -- 172.21.15.103:0/72695757 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 17) ==== 153302+0+0 (secure 0 0 0) 0x7f7c74005460 con 0x7f7c7c10a840 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.372+0000 7f7c7a000640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7c600b5410 0x7f7c600b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.372+0000 7f7c7a000640 1 -- 172.21.15.103:0/72695757 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7c740e67b0 con 0x7f7c7c10a840 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.372+0000 7f7c81000640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7c600b5410 0x7f7c600b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.373+0000 7f7c81000640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7c600b5410 0x7f7c600b78d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c000c00 tx=0x7f7c6c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:22:58.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.373+0000 7f7c7a000640 1 -- 172.21.15.103:0/72695757 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7c740ad180 con 0x7f7c7c10a840 2024-09-11T00:22:58.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.567+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7c7c063590 con 0x7f7c600b5410 2024-09-11T00:22:58.569 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:58 smithi103 ceph-mon[30578]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T00:22:58.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.580+0000 7f7c7a000640 1 -- 172.21.15.103:0/72695757 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f7c7c063590 con 0x7f7c600b5410 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:mds.a smithi103 running (2m) 21s ago 2m 13.2M - 19.3.0-4854-g8f219c47 589dd18248e4 6151c5100799 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:mds.b smithi117 running (2m) 31s ago 2m 26.9M - 19.3.0-4854-g8f219c47 589dd18248e4 0c50953b2ef5 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:mds.c smithi183 running (3m) 31s ago 3m 18.4M - 19.3.0-4854-g8f219c47 589dd18248e4 5c800929afb3 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:mds.d smithi103 running (3m) 21s ago 3m 13.3M - 19.3.0-4854-g8f219c47 589dd18248e4 da856ad467f4 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:mds.e smithi117 running (3m) 31s ago 3m 18.4M - 19.3.0-4854-g8f219c47 589dd18248e4 6baff9b7466e 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:mds.f smithi183 running (3m) 31s ago 3m 21.9M - 19.3.0-4854-g8f219c47 589dd18248e4 c8584de6ddb1 2024-09-11T00:22:58.582 INFO:teuthology.orchestra.run.smithi103.stdout:mds.g smithi103 running (2m) 21s ago 2m 13.3M - 19.3.0-4854-g8f219c47 589dd18248e4 c198b4477c4e 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mds.h smithi117 running (2m) 31s ago 2m 18.3M - 19.3.0-4854-g8f219c47 589dd18248e4 01b72f8cd469 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mds.i smithi183 running (2m) 31s ago 2m 21.6M - 19.3.0-4854-g8f219c47 589dd18248e4 7c771aa545ac 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mds.j smithi103 running (3m) 21s ago 3m 13.4M - 19.3.0-4854-g8f219c47 589dd18248e4 59160ec8d948 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mds.k smithi117 running (3m) 31s ago 3m 15.1M - 19.3.0-4854-g8f219c47 589dd18248e4 f4c80cfd8ddd 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mds.l smithi183 running (3m) 31s ago 3m 14.0M - 19.3.0-4854-g8f219c47 589dd18248e4 82cb105a7b6f 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.x smithi103 *:9283,8765 running (11m) 21s ago 11m 527M - 19.3.0-4854-g8f219c47 589dd18248e4 5762ba2c7558 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.y smithi117 *:8443,8765 running (9m) 31s ago 9m 458M - 19.3.0-4854-g8f219c47 589dd18248e4 c174358fee86 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mgr.z smithi183 *:8443,8765 running (9m) 31s ago 9m 459M - 19.3.0-4854-g8f219c47 589dd18248e4 81633643c467 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mon.a smithi103 running (11m) 21s ago 11m 52.2M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 0cabdefb1d1c 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mon.b smithi117 running (9m) 31s ago 9m 43.9M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 180a9c408211 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:mon.c smithi183 running (9m) 31s ago 9m 44.1M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 c3c8548ca275 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 smithi103 running (8m) 21s ago 8m 44.8M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 c4e18bd557be 2024-09-11T00:22:58.583 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 smithi117 running (8m) 31s ago 8m 56.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 3b3b2fa3f6bc 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 smithi183 running (7m) 31s ago 7m 57.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 1b6a34b7be7b 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 smithi103 running (7m) 21s ago 7m 43.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 012235ba4d2e 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 smithi117 running (6m) 31s ago 6m 54.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 9e2ba897770f 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 smithi183 running (6m) 31s ago 6m 44.1M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 19bccb2d639a 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 smithi103 running (5m) 21s ago 5m 42.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 873e73886f51 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 smithi117 running (5m) 31s ago 5m 52.8M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 4dde8bfa0d77 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 smithi183 running (5m) 31s ago 4m 53.1M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 3716f715ca77 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 smithi103 running (4m) 21s ago 4m 43.1M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 f4f29ec5dfb8 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 smithi117 running (4m) 31s ago 4m 44.9M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 1caf242f3e93 2024-09-11T00:22:58.584 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 smithi183 running (3m) 31s ago 3m 42.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 3c6da5c8bdca 2024-09-11T00:22:58.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7c600b5410 msgr2=0x7f7c600b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:58.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7c600b5410 0x7f7c600b78d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c000c00 tx=0x7f7c6c0314a0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c10a840 msgr2=0x7f7c7c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c10a840 0x7f7c7c19bff0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7c74005fd0 tx=0x7f7c7400ef10 comp rx=0 tx=0).stop 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 shutdown_connections 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7c600b5410 0x7f7c600b78d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c7c10a840 0x7f7c7c19bff0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c7c076f00 0x7f7c7c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 --2- 172.21.15.103:0/72695757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7c7c075aa0 0x7f7c7c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 >> 172.21.15.103:0/72695757 conn(0x7f7c7c0fe180 msgr2=0x7f7c7c106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 shutdown_connections 2024-09-11T00:22:58.586 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:22:58.582+0000 7f7c82400640 1 -- 172.21.15.103:0/72695757 wait complete. 2024-09-11T00:22:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:58 smithi183 ceph-mon[35934]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T00:22:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:58 smithi117 ceph-mon[35525]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-11T00:22:59.390 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T00:22:59.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:22:59 smithi103 ceph-mon[30578]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:22:59.770 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:22:59.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:22:59 smithi183 ceph-mon[35934]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:22:59.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:22:59 smithi117 ceph-mon[35525]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:00.406 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 -- 172.21.15.103:0/4007967809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 msgr2=0x7fad24102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 --2- 172.21.15.103:0/4007967809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 0x7fad24102c90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fad14009c40 tx=0x7fad1402a710 comp rx=0 tx=0).stop 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 -- 172.21.15.103:0/4007967809 shutdown_connections 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 --2- 172.21.15.103:0/4007967809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241043b0 0x7fad2410ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 --2- 172.21.15.103:0/4007967809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241039f0 0x7fad24103e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 --2- 172.21.15.103:0/4007967809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 0x7fad24102c90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 -- 172.21.15.103:0/4007967809 >> 172.21.15.103:0/4007967809 conn(0x7fad240fe180 msgr2=0x7fad241005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 -- 172.21.15.103:0/4007967809 shutdown_connections 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.367+0000 7fad23e00640 1 -- 172.21.15.103:0/4007967809 wait complete. 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 Processor -- start 2024-09-11T00:23:00.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 -- start start 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 0x7fad2419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241039f0 0x7fad2419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241043b0 0x7fad2419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410e4f0 con 0x7fad241039f0 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410e370 con 0x7fad241043b0 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.368+0000 7fad23e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad2410e670 con 0x7fad24102890 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad1a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241039f0 0x7fad2419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241043b0 0x7fad2419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad22a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 0x7fad2419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241043b0 0x7fad2419c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33928/0 (socket says 172.21.15.103:33928) 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad1a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241039f0 0x7fad2419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:54674/0 (socket says 172.21.15.103:54674) 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad22a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 0x7fad2419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59420/0 (socket says 172.21.15.103:59420) 2024-09-11T00:23:00.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad1a000640 1 -- 172.21.15.103:0/3006370665 learned_addr learned my addr 172.21.15.103:0/3006370665 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 -- 172.21.15.103:0/3006370665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 msgr2=0x7fad2419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 0x7fad2419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 -- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241039f0 msgr2=0x7fad2419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241039f0 0x7fad2419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 -- 172.21.15.103:0/3006370665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2419c920 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad1a000640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241039f0 0x7fad2419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.369+0000 7fad23400640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241043b0 0x7fad2419c080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad1c005fd0 tx=0x7fad1c00ed20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.370+0000 7fad1be00640 1 -- 172.21.15.103:0/3006370665 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00ccc0 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.370+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2419cc10 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.370+0000 7fad1be00640 1 -- 172.21.15.103:0/3006370665 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1c0035f0 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.370+0000 7fad1be00640 1 -- 172.21.15.103:0/3006370665 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00ce60 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.370+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad241a32f0 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.371+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad241a34c0 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.373+0000 7fad1be00640 1 -- 172.21.15.103:0/3006370665 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fad1c004000 con 0x7fad241043b0 2024-09-11T00:23:00.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.374+0000 7fad1be00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fad080b5390 0x7fad080b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.375+0000 7fad1be00640 1 -- 172.21.15.103:0/3006370665 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fad1c0e6320 con 0x7fad241043b0 2024-09-11T00:23:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.375+0000 7fad22a00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fad080b5390 0x7fad080b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.375+0000 7fad22a00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fad080b5390 0x7fad080b7850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad1402ad00 tx=0x7fad14000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:00.410 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.376+0000 7fad1be00640 1 -- 172.21.15.103:0/3006370665 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fad1c009510 con 0x7fad241043b0 2024-09-11T00:23:00.579 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.577+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fad24076aa0 con 0x7fad080b5390 2024-09-11T00:23:00.579 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:00 smithi103 ceph-mon[30578]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-11T00:23:00.579 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:00 smithi103 ceph-mon[30578]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:00.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.583+0000 7fad1be00640 1 -- 172.21.15.103:0/3006370665 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fad24076aa0 con 0x7fad080b5390 2024-09-11T00:23:00.585 INFO:teuthology.orchestra.run.smithi103.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T00:23:00.585 INFO:teuthology.orchestra.run.smithi103.stdout:mds.all 12/12 33s ago 3m smithi103=a;smithi103=d;smithi103=g;smithi103=j;smithi117=b;smithi117=e;smithi117=h;smithi117=k;smithi183=c;smithi183=f;smithi183=i;smithi183=l;count:12 2024-09-11T00:23:00.585 INFO:teuthology.orchestra.run.smithi103.stdout:mgr 3/3 33s ago 9m smithi103=x;smithi117=y;smithi183=z;count:3 2024-09-11T00:23:00.585 INFO:teuthology.orchestra.run.smithi103.stdout:mon 3/3 33s ago 9m smithi103:172.21.15.103=a;smithi117:172.21.15.117=b;smithi183:172.21.15.183=c;count:3 2024-09-11T00:23:00.585 INFO:teuthology.orchestra.run.smithi103.stdout:osd 12 33s ago - 2024-09-11T00:23:00.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fad080b5390 msgr2=0x7fad080b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:00.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fad080b5390 0x7fad080b7850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad1402ad00 tx=0x7fad14000f40 comp rx=0 tx=0).stop 2024-09-11T00:23:00.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241043b0 msgr2=0x7fad2419c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:00.587 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241043b0 0x7fad2419c080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fad1c005fd0 tx=0x7fad1c00ed20 comp rx=0 tx=0).stop 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 shutdown_connections 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fad080b5390 0x7fad080b7850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad241043b0 0x7fad2419c080 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad241039f0 0x7fad2419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 --2- 172.21.15.103:0/3006370665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad24102890 0x7fad2419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 >> 172.21.15.103:0/3006370665 conn(0x7fad240fe180 msgr2=0x7fad24075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 shutdown_connections 2024-09-11T00:23:00.588 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:00.585+0000 7fad23e00640 1 -- 172.21.15.103:0/3006370665 wait complete. 2024-09-11T00:23:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:00 smithi183 ceph-mon[35934]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-11T00:23:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:00 smithi183 ceph-mon[35934]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:00 smithi117 ceph-mon[35525]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-09-11T00:23:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:00 smithi117 ceph-mon[35525]: pgmap v396: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:01.535 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T00:23:01.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:01 smithi183 ceph-mon[35934]: from='client.25160 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:01.905 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:01.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:01 smithi103 ceph-mon[30578]: from='client.25160 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:01.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:01 smithi117 ceph-mon[35525]: from='client.25160 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:02.624 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:02 smithi103 ceph-mon[30578]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:02.628 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 -- 172.21.15.103:0/2634835809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34103b90 msgr2=0x7feb34104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 --2- 172.21.15.103:0/2634835809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34103b90 0x7feb34104010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb200098c0 tx=0x7feb2002a390 comp rx=0 tx=0).stop 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 -- 172.21.15.103:0/2634835809 shutdown_connections 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 --2- 172.21.15.103:0/2634835809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34104550 0x7feb3410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 --2- 172.21.15.103:0/2634835809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34103b90 0x7feb34104010 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 --2- 172.21.15.103:0/2634835809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 0x7feb34102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 -- 172.21.15.103:0/2634835809 >> 172.21.15.103:0/2634835809 conn(0x7feb340fe320 msgr2=0x7feb34100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 -- 172.21.15.103:0/2634835809 shutdown_connections 2024-09-11T00:23:02.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.626+0000 7feb39a00640 1 -- 172.21.15.103:0/2634835809 wait complete. 2024-09-11T00:23:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.627+0000 7feb39a00640 1 Processor -- start 2024-09-11T00:23:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.627+0000 7feb39a00640 1 -- start start 2024-09-11T00:23:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb39a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 0x7feb34076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb39a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34103b90 0x7feb34075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb39a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34104550 0x7feb34075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb39a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410e690 con 0x7feb34103b90 2024-09-11T00:23:02.630 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb39a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410e510 con 0x7feb34102a30 2024-09-11T00:23:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb39a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410e810 con 0x7feb34104550 2024-09-11T00:23:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb32a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 0x7feb34076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb32000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34103b90 0x7feb34075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb32a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 0x7feb34076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:54438/0 (socket says 172.21.15.103:54438) 2024-09-11T00:23:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb32a00640 1 -- 172.21.15.103:0/2841533537 learned_addr learned my addr 172.21.15.103:0/2841533537 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb32000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34103b90 0x7feb34075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42090/0 (socket says 172.21.15.103:42090) 2024-09-11T00:23:02.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.628+0000 7feb33400640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34104550 0x7feb34075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb32a00640 1 -- 172.21.15.103:0/2841533537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34104550 msgr2=0x7feb34075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb32a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34104550 0x7feb34075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb32a00640 1 -- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34103b90 msgr2=0x7feb34075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb32a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34103b90 0x7feb34075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb32a00640 1 -- 172.21.15.103:0/2841533537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb34076340 con 0x7feb34102a30 2024-09-11T00:23:02.632 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb32a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 0x7feb34076cd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e9f0 tx=0x7feb2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:02.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb2b400640 1 -- 172.21.15.103:0/2841533537 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2400cd80 con 0x7feb34102a30 2024-09-11T00:23:02.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb2b400640 1 -- 172.21.15.103:0/2841533537 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb240108e0 con 0x7feb34102a30 2024-09-11T00:23:02.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341a2f30 con 0x7feb34102a30 2024-09-11T00:23:02.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb2b400640 1 -- 172.21.15.103:0/2841533537 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24018980 con 0x7feb34102a30 2024-09-11T00:23:02.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.629+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341a3380 con 0x7feb34102a30 2024-09-11T00:23:02.633 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.631+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb341a35b0 con 0x7feb34102a30 2024-09-11T00:23:02.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.632+0000 7feb2b400640 1 -- 172.21.15.103:0/2841533537 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7feb24010430 con 0x7feb34102a30 2024-09-11T00:23:02.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.634+0000 7feb2b400640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feb180b52e0 0x7feb180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:02.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.634+0000 7feb2b400640 1 -- 172.21.15.103:0/2841533537 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feb240e62b0 con 0x7feb34102a30 2024-09-11T00:23:02.636 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.634+0000 7feb32000640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feb180b52e0 0x7feb180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:02.637 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.635+0000 7feb32000640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feb180b52e0 0x7feb180b77a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb200304b0 tx=0x7feb20005040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:02.639 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.637+0000 7feb2b400640 1 -- 172.21.15.103:0/2841533537 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7feb240ac730 con 0x7feb34102a30 2024-09-11T00:23:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:02 smithi183 ceph-mon[35934]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:02.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.835+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7feb34105680 con 0x7feb180b52e0 2024-09-11T00:23:02.838 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.836+0000 7feb2b400640 1 -- 172.21.15.103:0/2841533537 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7feb34105680 con 0x7feb180b52e0 2024-09-11T00:23:02.838 INFO:teuthology.orchestra.run.smithi103.stdout:HOST ADDR LABELS STATUS 2024-09-11T00:23:02.838 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 172.21.15.103 2024-09-11T00:23:02.838 INFO:teuthology.orchestra.run.smithi103.stdout:smithi117 172.21.15.117 2024-09-11T00:23:02.838 INFO:teuthology.orchestra.run.smithi103.stdout:smithi183 172.21.15.183 2024-09-11T00:23:02.839 INFO:teuthology.orchestra.run.smithi103.stdout:3 hosts in cluster 2024-09-11T00:23:02.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feb180b52e0 msgr2=0x7feb180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:02.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feb180b52e0 0x7feb180b77a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb200304b0 tx=0x7feb20005040 comp rx=0 tx=0).stop 2024-09-11T00:23:02.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 msgr2=0x7feb34076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:02.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 0x7feb34076cd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e9f0 tx=0x7feb2400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 shutdown_connections 2024-09-11T00:23:02.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feb180b52e0 0x7feb180b77a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.841 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb34104550 0x7feb34075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb34103b90 0x7feb34075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 --2- 172.21.15.103:0/2841533537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34102a30 0x7feb34076cd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:02.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 >> 172.21.15.103:0/2841533537 conn(0x7feb340fe320 msgr2=0x7feb34104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:02.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 shutdown_connections 2024-09-11T00:23:02.842 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:02.839+0000 7feb39a00640 1 -- 172.21.15.103:0/2841533537 wait complete. 2024-09-11T00:23:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:02 smithi117 ceph-mon[35525]: pgmap v397: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:03.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:03 smithi103 ceph-mon[30578]: from='client.25166 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:03.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:03 smithi183 ceph-mon[35934]: from='client.25166 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:03.806 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T00:23:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:03 smithi117 ceph-mon[35525]: from='client.25166 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:04.185 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:04.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:04 smithi103 ceph-mon[30578]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:04 smithi183 ceph-mon[35934]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.802+0000 7fe71e400640 1 -- 172.21.15.103:0/4252968762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718104550 msgr2=0x7fe71810aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.802+0000 7fe71e400640 1 --2- 172.21.15.103:0/4252968762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718104550 0x7fe71810aed0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe710009cd0 tx=0x7fe71002a8e0 comp rx=0 tx=0).stop 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.803+0000 7fe71e400640 1 -- 172.21.15.103:0/4252968762 shutdown_connections 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.803+0000 7fe71e400640 1 --2- 172.21.15.103:0/4252968762 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718104550 0x7fe71810aed0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.803+0000 7fe71e400640 1 --2- 172.21.15.103:0/4252968762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe718103b90 0x7fe718104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.803+0000 7fe71e400640 1 --2- 172.21.15.103:0/4252968762 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718102a30 0x7fe718102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.803+0000 7fe71e400640 1 -- 172.21.15.103:0/4252968762 >> 172.21.15.103:0/4252968762 conn(0x7fe7180fe320 msgr2=0x7fe718100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:04.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.803+0000 7fe71e400640 1 -- 172.21.15.103:0/4252968762 shutdown_connections 2024-09-11T00:23:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.803+0000 7fe71e400640 1 -- 172.21.15.103:0/4252968762 wait complete. 2024-09-11T00:23:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 Processor -- start 2024-09-11T00:23:04.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 -- start start 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718102a30 0x7fe71819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe718103b90 0x7fe71819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718104550 0x7fe71819c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe71810e690 con 0x7fe718102a30 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe71810e510 con 0x7fe718104550 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.804+0000 7fe71e400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe71810e810 con 0x7fe718103b90 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718102a30 0x7fe71819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718104550 0x7fe71819c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe70ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe718103b90 0x7fe71819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718104550 0x7fe71819c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:54460/0 (socket says 172.21.15.103:54460) 2024-09-11T00:23:04.807 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 -- 172.21.15.103:0/3498589574 learned_addr learned my addr 172.21.15.103:0/3498589574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:04.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 -- 172.21.15.103:0/3498589574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe718103b90 msgr2=0x7fe71819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:04.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe718103b90 0x7fe71819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:04.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 -- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718102a30 msgr2=0x7fe71819b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:04.808 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718102a30 0x7fe71819b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 -- 172.21.15.103:0/3498589574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71819cb70 con 0x7fe718104550 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.805+0000 7fe717e00640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718104550 0x7fe71819c2d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe7100317d0 tx=0x7fe710003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.806+0000 7fe714c00640 1 -- 172.21.15.103:0/3498589574 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710004250 con 0x7fe718104550 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.806+0000 7fe714c00640 1 -- 172.21.15.103:0/3498589574 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7100305a0 con 0x7fe718104550 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.806+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71819ce00 con 0x7fe718104550 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.806+0000 7fe714c00640 1 -- 172.21.15.103:0/3498589574 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710045670 con 0x7fe718104550 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.806+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7181a3540 con 0x7fe718104550 2024-09-11T00:23:04.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.807+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7181a3710 con 0x7fe718104550 2024-09-11T00:23:04.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.809+0000 7fe714c00640 1 -- 172.21.15.103:0/3498589574 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe71004d070 con 0x7fe718104550 2024-09-11T00:23:04.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.810+0000 7fe714c00640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6fc0b5360 0x7fe6fc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:04.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.810+0000 7fe714c00640 1 -- 172.21.15.103:0/3498589574 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe71002e030 con 0x7fe718104550 2024-09-11T00:23:04.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.810+0000 7fe717400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6fc0b5360 0x7fe6fc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:04.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.811+0000 7fe717400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6fc0b5360 0x7fe6fc0b7820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe708009bd0 tx=0x7fe708005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:04.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:04.812+0000 7fe714c00640 1 -- 172.21.15.103:0/3498589574 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe71004a060 con 0x7fe718104550 2024-09-11T00:23:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:04 smithi117 ceph-mon[35525]: pgmap v398: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:05.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.003+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fe718076aa0 con 0x7fe6fc0b5360 2024-09-11T00:23:05.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.005+0000 7fe714c00640 1 -- 172.21.15.103:0/3498589574 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fe718076aa0 con 0x7fe6fc0b5360 2024-09-11T00:23:05.007 INFO:teuthology.orchestra.run.smithi103.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T00:23:05.007 INFO:teuthology.orchestra.run.smithi103.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-11T00:23:05.007 INFO:teuthology.orchestra.run.smithi103.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-11T00:23:05.007 INFO:teuthology.orchestra.run.smithi103.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.007+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6fc0b5360 msgr2=0x7fe6fc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.007+0000 7fe71e400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6fc0b5360 0x7fe6fc0b7820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe708009bd0 tx=0x7fe708005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718104550 msgr2=0x7fe71819c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718104550 0x7fe71819c2d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe7100317d0 tx=0x7fe710003c10 comp rx=0 tx=0).stop 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 shutdown_connections 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6fc0b5360 0x7fe6fc0b7820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe718104550 0x7fe71819c2d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe718103b90 0x7fe71819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 --2- 172.21.15.103:0/3498589574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe718102a30 0x7fe71819b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 >> 172.21.15.103:0/3498589574 conn(0x7fe7180fe320 msgr2=0x7fe718075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 shutdown_connections 2024-09-11T00:23:05.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:05.008+0000 7fe71e400640 1 -- 172.21.15.103:0/3498589574 wait complete. 2024-09-11T00:23:05.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:05 smithi183 ceph-mon[35934]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:05.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:05 smithi103 ceph-mon[30578]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:05.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:05 smithi117 ceph-mon[35525]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T00:23:05.989 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T00:23:06.003 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:23:06.003 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T00:23:06.374 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:06.776 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:06 smithi103 ceph-mon[30578]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:06.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:06 smithi117 ceph-mon[35525]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:07.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:06 smithi183 ceph-mon[35934]: pgmap v399: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.113+0000 7fcf99c00640 1 -- 172.21.15.103:0/3658090532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 msgr2=0x7fcf941069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.113+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3658090532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 0x7fcf941069f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009ca0 tx=0x7fcf8c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.113+0000 7fcf99c00640 1 -- 172.21.15.103:0/3658090532 shutdown_connections 2024-09-11T00:23:07.178 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.113+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3658090532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 0x7fcf941069f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.113+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3658090532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94103990 0x7fcf94105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.113+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3658090532 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 0x7fcf94103400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.113+0000 7fcf99c00640 1 -- 172.21.15.103:0/3658090532 >> 172.21.15.103:0/3658090532 conn(0x7fcf940fd220 msgr2=0x7fcf940ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.114+0000 7fcf99c00640 1 -- 172.21.15.103:0/3658090532 shutdown_connections 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.114+0000 7fcf99c00640 1 -- 172.21.15.103:0/3658090532 wait complete. 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 Processor -- start 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 -- start start 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 0x7fcf9419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94103990 0x7fcf9419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:07.179 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 0x7fcf941a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94114d10 con 0x7fcf94106540 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94114b90 con 0x7fcf94103990 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.115+0000 7fcf99c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94114e90 con 0x7fcf9406c140 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 0x7fcf9419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 0x7fcf9419f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:34778/0 (socket says 172.21.15.103:34778) 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 -- 172.21.15.103:0/3830347495 learned_addr learned my addr 172.21.15.103:0/3830347495 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92000640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94103990 0x7fcf9419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf93400640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 0x7fcf941a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 -- 172.21.15.103:0/3830347495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94103990 msgr2=0x7fcf9419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:07.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94103990 0x7fcf9419fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 -- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 msgr2=0x7fcf941a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 0x7fcf941a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 -- 172.21.15.103:0/3830347495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf941a0f20 con 0x7fcf9406c140 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.116+0000 7fcf92a00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 0x7fcf9419f3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400e9f0 tx=0x7fcf8400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.117+0000 7fcf8b400640 1 -- 172.21.15.103:0/3830347495 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400cd80 con 0x7fcf9406c140 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.117+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf941a1210 con 0x7fcf9406c140 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.117+0000 7fcf8b400640 1 -- 172.21.15.103:0/3830347495 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf840054d0 con 0x7fcf9406c140 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.117+0000 7fcf8b400640 1 -- 172.21.15.103:0/3830347495 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84010690 con 0x7fcf9406c140 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.117+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941a78f0 con 0x7fcf9406c140 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.118+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf941069f0 con 0x7fcf9406c140 2024-09-11T00:23:07.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.120+0000 7fcf8b400640 1 -- 172.21.15.103:0/3830347495 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcf84004e30 con 0x7fcf9406c140 2024-09-11T00:23:07.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.122+0000 7fcf8b400640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52e0 0x7fcf780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:07.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.122+0000 7fcf8b400640 1 -- 172.21.15.103:0/3830347495 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcf840e65a0 con 0x7fcf9406c140 2024-09-11T00:23:07.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.122+0000 7fcf92000640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52e0 0x7fcf780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:07.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.123+0000 7fcf92000640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52e0 0x7fcf780b77a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf9406c5f0 tx=0x7fcf80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:07.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.123+0000 7fcf8b400640 1 -- 172.21.15.103:0/3830347495 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcf840ad070 con 0x7fcf9406c140 2024-09-11T00:23:07.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.345+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fcf9402e640 con 0x7fcf9406c140 2024-09-11T00:23:07.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.345+0000 7fcf8b400640 1 -- 172.21.15.103:0/3830347495 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) ==== 76+0+3102 (secure 0 0 0) 0x7fcf840ac970 con 0x7fcf9406c140 2024-09-11T00:23:07.348 INFO:teuthology.orchestra.run.smithi103.stdout:e26 2024-09-11T00:23:07.348 INFO:teuthology.orchestra.run.smithi103.stdout:btime 2024-09-11T00:21:28:210497+0000 2024-09-11T00:23:07.348 INFO:teuthology.orchestra.run.smithi103.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T00:23:07.348 INFO:teuthology.orchestra.run.smithi103.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:23:07.348 INFO:teuthology.orchestra.run.smithi103.stdout:legacy client fscid: 1 2024-09-11T00:23:07.348 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:07.348 INFO:teuthology.orchestra.run.smithi103.stdout:Filesystem 'cephfs' (1) 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:fs_name cephfs 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 26 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-11T00:20:28.229329+0000 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-11T00:21:27.247376+0000 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:tableserver 0 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:root 0 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:session_timeout 300 2024-09-11T00:23:07.349 INFO:teuthology.orchestra.run.smithi103.stdout:session_autoclose 300 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:max_file_size 1099511627776 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:max_xattr_size 65536 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:required_client_features {} 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure 0 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:last_failure_osd_epoch 0 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:max_mds 3 2024-09-11T00:23:07.350 INFO:teuthology.orchestra.run.smithi103.stdout:in 0,1,2 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:up {0=24656,1=24643,2=24628} 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:failed 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:damaged 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:stopped 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:data_pools [3,4] 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:metadata_pool 2 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:inline_data disabled 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:balancer 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:bal_rank_mask -1 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:standby_count_wanted 0 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:qdb_cluster leader: 24656 members: 24656,24643,24628 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b{0:24656} state up:active seq 5 addr [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.351 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.f{0:24622} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/3946403859,v1:172.21.15.183:6835/3946403859] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.i{1:24643} state up:active seq 21 addr [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.h{1:24620} state up:standby-replay seq 1 addr [v2:172.21.15.117:6836/2700270511,v1:172.21.15.117:6837/2700270511] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c{2:24628} state up:active seq 23 addr [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.e{2:24599} state up:standby-replay seq 1 addr [v2:172.21.15.117:6834/2973217634,v1:172.21.15.117:6835/2973217634] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout:Standby daemons: 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.j{-1:14580} state up:standby seq 1 addr [v2:172.21.15.103:6834/1632166659,v1:172.21.15.103:6835/1632166659] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.352 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d{-1:14592} state up:standby seq 1 addr [v2:172.21.15.103:6836/3157272512,v1:172.21.15.103:6837/3157272512] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.353 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.g{-1:14628} state up:standby seq 1 addr [v2:172.21.15.103:6838/3371048612,v1:172.21.15.103:6839/3371048612] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.353 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a{-1:14640} state up:standby seq 1 addr [v2:172.21.15.103:6840/4229734989,v1:172.21.15.103:6841/4229734989] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.353 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.k{-1:24566} state up:standby seq 1 addr [v2:172.21.15.117:6832/928029640,v1:172.21.15.117:6833/928029640] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.353 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.l{-1:24571} state up:standby seq 1 addr [v2:172.21.15.183:6832/1084194007,v1:172.21.15.183:6833/1084194007] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52e0 msgr2=0x7fcf780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52e0 0x7fcf780b77a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf9406c5f0 tx=0x7fcf80005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 msgr2=0x7fcf9419f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 0x7fcf9419f3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400e9f0 tx=0x7fcf8400edf0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 shutdown_connections 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcf780b52e0 0x7fcf780b77a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf94106540 0x7fcf941a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf94103990 0x7fcf9419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 --2- 172.21.15.103:0/3830347495 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf9406c140 0x7fcf9419f3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:07.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 >> 172.21.15.103:0/3830347495 conn(0x7fcf940fd220 msgr2=0x7fcf94104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:07.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 shutdown_connections 2024-09-11T00:23:07.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:07.348+0000 7fcf99c00640 1 -- 172.21.15.103:0/3830347495 wait complete. 2024-09-11T00:23:07.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:07 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3830347495' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:23:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:07 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3830347495' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:23:08.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:07 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3830347495' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-11T00:23:08.256 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T00:23:08.630 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:08.802 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:08 smithi103 ceph-mon[30578]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T00:23:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:08 smithi117 ceph-mon[35525]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T00:23:09.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:08 smithi183 ceph-mon[35934]: pgmap v400: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T00:23:09.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 -- 172.21.15.103:0/2096045479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4102a20 msgr2=0x7fc7b4102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 --2- 172.21.15.103:0/2096045479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4102a20 0x7fc7b4102e20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009930 tx=0x7fc7a402a420 comp rx=0 tx=0).stop 2024-09-11T00:23:09.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 -- 172.21.15.103:0/2096045479 shutdown_connections 2024-09-11T00:23:09.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 --2- 172.21.15.103:0/2096045479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7b4104540 0x7fc7b410aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 --2- 172.21.15.103:0/2096045479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7b4103b80 0x7fc7b4104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 --2- 172.21.15.103:0/2096045479 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4102a20 0x7fc7b4102e20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 -- 172.21.15.103:0/2096045479 >> 172.21.15.103:0/2096045479 conn(0x7fc7b40fe2f0 msgr2=0x7fc7b4100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 -- 172.21.15.103:0/2096045479 shutdown_connections 2024-09-11T00:23:09.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.264+0000 7fc7b9400640 1 -- 172.21.15.103:0/2096045479 wait complete. 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.265+0000 7fc7b9400640 1 Processor -- start 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b9400640 1 -- start start 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7b4102a20 0x7fc7b4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7b4103b80 0x7fc7b41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4104540 0x7fc7b419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b9400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410e6a0 con 0x7fc7b4104540 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b9400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410e520 con 0x7fc7b4102a20 2024-09-11T00:23:09.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b9400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b410e820 con 0x7fc7b4103b80 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.266+0000 7fc7b3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4104540 0x7fc7b419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4104540 0x7fc7b419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42146/0 (socket says 172.21.15.103:42146) 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b3200640 1 -- 172.21.15.103:0/3734034363 learned_addr learned my addr 172.21.15.103:0/3734034363 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b1e00640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7b4103b80 0x7fc7b41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b2800640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7b4102a20 0x7fc7b4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b2800640 1 -- 172.21.15.103:0/3734034363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7b4103b80 msgr2=0x7fc7b41994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b2800640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7b4103b80 0x7fc7b41994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b2800640 1 -- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4104540 msgr2=0x7fc7b419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b2800640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4104540 0x7fc7b419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b2800640 1 -- 172.21.15.103:0/3734034363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b419a960 con 0x7fc7b4102a20 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b3200640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4104540 0x7fc7b419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b1e00640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7b4103b80 0x7fc7b41994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:09.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.267+0000 7fc7b2800640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7b4102a20 0x7fc7b4198ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005740 tx=0x7fc7a40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.268+0000 7fc7ab400640 1 -- 172.21.15.103:0/3734034363 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a403f070 con 0x7fc7b4102a20 2024-09-11T00:23:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.268+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b419abf0 con 0x7fc7b4102a20 2024-09-11T00:23:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.268+0000 7fc7ab400640 1 -- 172.21.15.103:0/3734034363 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4032e40 con 0x7fc7b4102a20 2024-09-11T00:23:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.268+0000 7fc7ab400640 1 -- 172.21.15.103:0/3734034363 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4035070 con 0x7fc7b4102a20 2024-09-11T00:23:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.268+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a1160 con 0x7fc7b4102a20 2024-09-11T00:23:09.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.269+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b4109e20 con 0x7fc7b4102a20 2024-09-11T00:23:09.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.271+0000 7fc7ab400640 1 -- 172.21.15.103:0/3734034363 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc7a404b060 con 0x7fc7b4102a20 2024-09-11T00:23:09.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.273+0000 7fc7ab400640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7980b52e0 0x7fc7980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:09.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.273+0000 7fc7ab400640 1 -- 172.21.15.103:0/3734034363 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc7a4106890 con 0x7fc7b4102a20 2024-09-11T00:23:09.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.273+0000 7fc7b1e00640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7980b52e0 0x7fc7980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:09.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.273+0000 7fc7b1e00640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7980b52e0 0x7fc7980b77a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009fc0 tx=0x7fc7a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:09.276 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.274+0000 7fc7ab400640 1 -- 172.21.15.103:0/3734034363 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc7a40ccd10 con 0x7fc7b4102a20 2024-09-11T00:23:09.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.463+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc7b402e780 con 0x7fc7b4102a20 2024-09-11T00:23:09.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.465+0000 7fc7ab400640 1 -- 172.21.15.103:0/3734034363 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5666 (secure 0 0 0) 0x7fc7a40d1740 con 0x7fc7b4102a20 2024-09-11T00:23:09.467 INFO:teuthology.orchestra.run.smithi103.stdout:epoch 89 2024-09-11T00:23:09.467 INFO:teuthology.orchestra.run.smithi103.stdout:fsid 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:23:09.467 INFO:teuthology.orchestra.run.smithi103.stdout:created 2024-09-11T00:11:30.997477+0000 2024-09-11T00:23:09.467 INFO:teuthology.orchestra.run.smithi103.stdout:modified 2024-09-11T00:22:28.671420+0000 2024-09-11T00:23:09.467 INFO:teuthology.orchestra.run.smithi103.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T00:23:09.467 INFO:teuthology.orchestra.run.smithi103.stdout:crush_version 27 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:full_ratio 0.95 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:backfillfull_ratio 0.9 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:nearfull_ratio 0.85 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:require_min_compat_client luminous 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:min_compat_client luminous 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:require_osd_release squid 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:stretch_mode_enabled false 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-11T00:23:09.468 INFO:teuthology.orchestra.run.smithi103.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:max_osd 12 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6802/2122983422,v1:172.21.15.103:6803/2122983422] [v2:172.21.15.103:6804/2122983422,v1:172.21.15.103:6805/2122983422] exists,up b2d81a93-417a-401b-88db-d1512a1bb426 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6800/2016738362,v1:172.21.15.117:6801/2016738362] [v2:172.21.15.117:6802/2016738362,v1:172.21.15.117:6803/2016738362] exists,up 0a35cffa-4c5a-4c39-851e-563d54f29b49 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/4126324003,v1:172.21.15.183:6801/4126324003] [v2:172.21.15.183:6802/4126324003,v1:172.21.15.183:6803/4126324003] exists,up fca0072a-688a-4c07-90b9-bd4bb84aa15f 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6810/2768325174,v1:172.21.15.103:6811/2768325174] [v2:172.21.15.103:6812/2768325174,v1:172.21.15.103:6813/2768325174] exists,up 39a7f76f-3edd-4144-b4c4-a3e2ca84001d 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.4 up in weight 1 up_from 31 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6808/392780746,v1:172.21.15.117:6809/392780746] [v2:172.21.15.117:6810/392780746,v1:172.21.15.117:6811/392780746] exists,up 5f479678-1b4e-4d7b-9c3e-d2033337c993 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.5 up in weight 1 up_from 36 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/660429502,v1:172.21.15.183:6809/660429502] [v2:172.21.15.183:6810/660429502,v1:172.21.15.183:6811/660429502] exists,up 3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.6 up in weight 1 up_from 42 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6818/962000049,v1:172.21.15.103:6819/962000049] [v2:172.21.15.103:6820/962000049,v1:172.21.15.103:6821/962000049] exists,up 7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.7 up in weight 1 up_from 47 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6816/2935555111,v1:172.21.15.117:6817/2935555111] [v2:172.21.15.117:6818/2935555111,v1:172.21.15.117:6819/2935555111] exists,up 504cd71d-934e-4a6e-9ed2-5fc4a758800e 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.8 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/3906862108,v1:172.21.15.183:6817/3906862108] [v2:172.21.15.183:6818/3906862108,v1:172.21.15.183:6819/3906862108] exists,up 0ede29a2-c541-4834-b320-cd6f66b7846a 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.9 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6826/64454554,v1:172.21.15.103:6827/64454554] [v2:172.21.15.103:6828/64454554,v1:172.21.15.103:6829/64454554] exists,up 7999315b-3589-4be7-9d06-4436f21e3235 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.10 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.117:6824/2500948230,v1:172.21.15.117:6825/2500948230] [v2:172.21.15.117:6826/2500948230,v1:172.21.15.117:6827/2500948230] exists,up bde15347-8049-49ab-a4e7-06a5cba2d225 2024-09-11T00:23:09.469 INFO:teuthology.orchestra.run.smithi103.stdout:osd.11 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/405835574,v1:172.21.15.183:6825/405835574] [v2:172.21.15.183:6826/405835574,v1:172.21.15.183:6827/405835574] exists,up 1bdface1-a9b3-4175-93ea-db2997b3c55b 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.4 [2,11] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 2.10 [8,11] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.0 [5,9] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.6 [3,7] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.17 [4,6] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.31 [4,8] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.33 [5,9] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 3.36 [4,9] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.e [5,8] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.10 [9,8] 2024-09-11T00:23:09.470 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.13 [9,10] 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:pg_upmap_items 4.1b [9,8] 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/489365644 expires 2024-09-12T00:12:26.257361+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1436162684 expires 2024-09-12T00:12:26.257361+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/2651148903 expires 2024-09-12T00:12:26.257361+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/817530946 expires 2024-09-12T00:12:26.257361+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/2651148903 expires 2024-09-12T00:12:26.257361+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/1510088401 expires 2024-09-12T00:11:57.620215+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6801/477475422 expires 2024-09-12T00:11:57.620215+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:6800/477475422 expires 2024-09-12T00:11:57.620215+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/3506454988 expires 2024-09-12T00:11:57.620215+0000 2024-09-11T00:23:09.471 INFO:teuthology.orchestra.run.smithi103.stdout:blocklist 172.21.15.103:0/2690135380 expires 2024-09-12T00:11:57.620215+0000 2024-09-11T00:23:09.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7980b52e0 msgr2=0x7fc7980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7980b52e0 0x7fc7980b77a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009fc0 tx=0x7fc7a0005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:09.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7b4102a20 msgr2=0x7fc7b4198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7b4102a20 0x7fc7b4198ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4005740 tx=0x7fc7a40042f0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 shutdown_connections 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7980b52e0 0x7fc7980b77a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b4104540 0x7fc7b419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc7b4103b80 0x7fc7b41994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 --2- 172.21.15.103:0/3734034363 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7b4102a20 0x7fc7b4198ea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 >> 172.21.15.103:0/3734034363 conn(0x7fc7b40fe2f0 msgr2=0x7fc7b4106ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 shutdown_connections 2024-09-11T00:23:09.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:09.467+0000 7fc7b9400640 1 -- 172.21.15.103:0/3734034363 wait complete. 2024-09-11T00:23:09.843 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:09 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3734034363' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:23:09.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:09 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3734034363' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:23:10.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:09 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3734034363' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-11T00:23:10.384 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T00:23:10.397 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:23:10.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:10 smithi103 ceph-mon[30578]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:10.773 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:10 smithi117 ceph-mon[35525]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:11.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:10 smithi183 ceph-mon[35934]: pgmap v401: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-11T00:23:11.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 -- 172.21.15.103:0/1230654020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8102c10 msgr2=0x7fe9c8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:11.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 --2- 172.21.15.103:0/1230654020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8102c10 0x7fe9c8102ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009900 tx=0x7fe9b802a410 comp rx=0 tx=0).stop 2024-09-11T00:23:11.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 -- 172.21.15.103:0/1230654020 shutdown_connections 2024-09-11T00:23:11.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 --2- 172.21.15.103:0/1230654020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c8108fa0 0x7fe9c810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 --2- 172.21.15.103:0/1230654020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8108620 0x7fe9c8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 --2- 172.21.15.103:0/1230654020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8102c10 0x7fe9c8102ff0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 -- 172.21.15.103:0/1230654020 >> 172.21.15.103:0/1230654020 conn(0x7fe9c80fe6c0 msgr2=0x7fe9c8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:11.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 -- 172.21.15.103:0/1230654020 shutdown_connections 2024-09-11T00:23:11.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.381+0000 7fe9c7200640 1 -- 172.21.15.103:0/1230654020 wait complete. 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 Processor -- start 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 -- start start 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8102c10 0x7fe9c819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c8108620 0x7fe9c819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8108fa0 0x7fe9c81a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8112c60 con 0x7fe9c8108fa0 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8112ae0 con 0x7fe9c8102c10 2024-09-11T00:23:11.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.382+0000 7fe9c7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8112de0 con 0x7fe9c8108620 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8102c10 0x7fe9c819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8108fa0 0x7fe9c81a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8102c10 0x7fe9c819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:54504/0 (socket says 172.21.15.103:54504) 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 -- 172.21.15.103:0/975111955 learned_addr learned my addr 172.21.15.103:0/975111955 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5400640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c8108620 0x7fe9c819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 -- 172.21.15.103:0/975111955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c8108620 msgr2=0x7fe9c819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c8108620 0x7fe9c819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 -- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8108fa0 msgr2=0x7fe9c81a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8108fa0 0x7fe9c81a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5e00640 1 -- 172.21.15.103:0/975111955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c81a1300 con 0x7fe9c8102c10 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c5400640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c8108620 0x7fe9c819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:11.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.383+0000 7fe9c6800640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8108fa0 0x7fe9c81a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:11.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.384+0000 7fe9c5e00640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8102c10 0x7fe9c819b250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8005940 tx=0x7fe9b8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:11.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.384+0000 7fe9bea00640 1 -- 172.21.15.103:0/975111955 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b803f030 con 0x7fe9c8102c10 2024-09-11T00:23:11.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.384+0000 7fe9bea00640 1 -- 172.21.15.103:0/975111955 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9b8003aa0 con 0x7fe9c8102c10 2024-09-11T00:23:11.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.384+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c81a1590 con 0x7fe9c8102c10 2024-09-11T00:23:11.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.384+0000 7fe9bea00640 1 -- 172.21.15.103:0/975111955 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8035070 con 0x7fe9c8102c10 2024-09-11T00:23:11.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.384+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c81a7cb0 con 0x7fe9c8102c10 2024-09-11T00:23:11.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.385+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c8102ff0 con 0x7fe9c8102c10 2024-09-11T00:23:11.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.387+0000 7fe9bea00640 1 -- 172.21.15.103:0/975111955 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe9b8003c50 con 0x7fe9c8102c10 2024-09-11T00:23:11.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.389+0000 7fe9bea00640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe9ac0b5390 0x7fe9ac0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:11.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.389+0000 7fe9bea00640 1 -- 172.21.15.103:0/975111955 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe9b8106b30 con 0x7fe9c8102c10 2024-09-11T00:23:11.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.389+0000 7fe9c5400640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe9ac0b5390 0x7fe9ac0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:11.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.390+0000 7fe9c5400640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe9ac0b5390 0x7fe9ac0b7850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8102050 tx=0x7fe9b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:11.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.391+0000 7fe9bea00640 1 -- 172.21.15.103:0/975111955 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe9b80ccfb0 con 0x7fe9c8102c10 2024-09-11T00:23:11.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.609+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9c81a7f10 con 0x7fe9c8102c10 2024-09-11T00:23:11.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.610+0000 7fe9bea00640 1 -- 172.21.15.103:0/975111955 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7fe9b80d19e0 con 0x7fe9c8102c10 2024-09-11T00:23:11.613 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:11.613 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-11T00:21:28:210497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:21:27.247376+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:23:11.613 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-11T00:23:11.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe9ac0b5390 msgr2=0x7fe9ac0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:11.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe9ac0b5390 0x7fe9ac0b7850 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8102050 tx=0x7fe9b4005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:11.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8102c10 msgr2=0x7fe9c819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:11.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8102c10 0x7fe9c819b250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8005940 tx=0x7fe9b8004270 comp rx=0 tx=0).stop 2024-09-11T00:23:11.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 shutdown_connections 2024-09-11T00:23:11.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe9ac0b5390 0x7fe9ac0b7850 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9c8108fa0 0x7fe9c81a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9c8108620 0x7fe9c819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 --2- 172.21.15.103:0/975111955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9c8102c10 0x7fe9c819b250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:11.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 >> 172.21.15.103:0/975111955 conn(0x7fe9c80fe6c0 msgr2=0x7fe9c810da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:11.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 shutdown_connections 2024-09-11T00:23:11.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:11.613+0000 7fe9c7200640 1 -- 172.21.15.103:0/975111955 wait complete. 2024-09-11T00:23:11.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:11 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/975111955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:11.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:11 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/975111955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:12.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:11 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/975111955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:12.441 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:23:12.811 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:12.932 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:12 smithi183 ceph-mon[35934]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:12.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:12 smithi117 ceph-mon[35525]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:12.949 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:12 smithi103 ceph-mon[30578]: pgmap v402: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:13.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 -- 172.21.15.103:0/3301245615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d2410b180 msgr2=0x7f3d24111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:13.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 --2- 172.21.15.103:0/3301245615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d2410b180 0x7f3d24111980 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009ca0 tx=0x7f3d1c02a8b0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 -- 172.21.15.103:0/3301245615 shutdown_connections 2024-09-11T00:23:13.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 --2- 172.21.15.103:0/3301245615 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d2410b180 0x7f3d24111980 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 --2- 172.21.15.103:0/3301245615 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2410a800 0x7f3d2410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 --2- 172.21.15.103:0/3301245615 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d24104df0 0x7f3d241051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 -- 172.21.15.103:0/3301245615 >> 172.21.15.103:0/3301245615 conn(0x7f3d24100840 msgr2=0x7f3d24102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:13.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 -- 172.21.15.103:0/3301245615 shutdown_connections 2024-09-11T00:23:13.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.432+0000 7f3d29a00640 1 -- 172.21.15.103:0/3301245615 wait complete. 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.433+0000 7f3d29a00640 1 Processor -- start 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.433+0000 7f3d29a00640 1 -- start start 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d29a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d24104df0 0x7f3d2419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d29a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2410a800 0x7f3d2419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d29a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2410b180 0x7f3d241a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d29a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d24114e90 con 0x7f3d24104df0 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d29a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d24114d10 con 0x7f3d2410b180 2024-09-11T00:23:13.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d29a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d24115010 con 0x7f3d2410a800 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d24104df0 0x7f3d2419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d24104df0 0x7f3d2419b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33262/0 (socket says 172.21.15.103:33262) 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 -- 172.21.15.103:0/2744008137 learned_addr learned my addr 172.21.15.103:0/2744008137 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23400640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2410a800 0x7f3d2419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d29000640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2410b180 0x7f3d241a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 -- 172.21.15.103:0/2744008137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2410a800 msgr2=0x7f3d2419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2410a800 0x7f3d2419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 -- 172.21.15.103:0/2744008137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2410b180 msgr2=0x7f3d241a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2410b180 0x7f3d241a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.434+0000 7f3d23e00640 1 -- 172.21.15.103:0/2744008137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d241a12d0 con 0x7f3d24104df0 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.435+0000 7f3d23e00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d24104df0 0x7f3d2419b220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9f0 tx=0x7f3d1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.435+0000 7f3d20c00640 1 -- 172.21.15.103:0/2744008137 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1400cd80 con 0x7f3d24104df0 2024-09-11T00:23:13.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.435+0000 7f3d20c00640 1 -- 172.21.15.103:0/2744008137 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d140054d0 con 0x7f3d24104df0 2024-09-11T00:23:13.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.435+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d241a15c0 con 0x7f3d24104df0 2024-09-11T00:23:13.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.435+0000 7f3d20c00640 1 -- 172.21.15.103:0/2744008137 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14010690 con 0x7f3d24104df0 2024-09-11T00:23:13.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.435+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d241a7c80 con 0x7f3d24104df0 2024-09-11T00:23:13.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.437+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d241051d0 con 0x7f3d24104df0 2024-09-11T00:23:13.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.438+0000 7f3d20c00640 1 -- 172.21.15.103:0/2744008137 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3d14004e90 con 0x7f3d24104df0 2024-09-11T00:23:13.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.441+0000 7f3d20c00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d080b5410 0x7f3d080b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:13.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.441+0000 7f3d20c00640 1 -- 172.21.15.103:0/2744008137 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3d140e63a0 con 0x7f3d24104df0 2024-09-11T00:23:13.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.441+0000 7f3d23400640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d080b5410 0x7f3d080b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:13.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.442+0000 7f3d23400640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d080b5410 0x7f3d080b78d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d241030b0 tx=0x7f3d10005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:13.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.444+0000 7f3d20c00640 1 -- 172.21.15.103:0/2744008137 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3d140ace70 con 0x7f3d24104df0 2024-09-11T00:23:13.668 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.666+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d241a7f40 con 0x7f3d24104df0 2024-09-11T00:23:13.669 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.667+0000 7f3d20c00640 1 -- 172.21.15.103:0/2744008137 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11933 (secure 0 0 0) 0x7f3d140ac770 con 0x7f3d24104df0 2024-09-11T00:23:13.669 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:13.670 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":26,"btime":"2024-09-11T00:21:28:210497+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:21:27.247376+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:23:13.670 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 26 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d080b5410 msgr2=0x7f3d080b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d080b5410 0x7f3d080b78d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d241030b0 tx=0x7f3d10005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d24104df0 msgr2=0x7f3d2419b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d24104df0 0x7f3d2419b220 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400e9f0 tx=0x7f3d1400edf0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 shutdown_connections 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d080b5410 0x7f3d080b78d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d2410b180 0x7f3d241a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.672 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d2410a800 0x7f3d2419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 --2- 172.21.15.103:0/2744008137 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d24104df0 0x7f3d2419b220 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:13.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 >> 172.21.15.103:0/2744008137 conn(0x7f3d24100840 msgr2=0x7f3d241065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:13.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 shutdown_connections 2024-09-11T00:23:13.673 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:13.670+0000 7f3d29a00640 1 -- 172.21.15.103:0/2744008137 wait complete. 2024-09-11T00:23:13.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:13 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2744008137' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:13.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:13 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2744008137' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:14.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:13 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2744008137' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:14.479 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:23:14.856 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:14.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:14 smithi103 ceph-mon[30578]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:14.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:14 smithi117 ceph-mon[35525]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:15.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:14 smithi183 ceph-mon[35934]: pgmap v403: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:15.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 -- 172.21.15.103:0/3935412590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 msgr2=0x7fed041059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:15.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 --2- 172.21.15.103:0/3935412590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 0x7fed041059b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009900 tx=0x7fecf002a410 comp rx=0 tx=0).stop 2024-09-11T00:23:15.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 -- 172.21.15.103:0/3935412590 shutdown_connections 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 --2- 172.21.15.103:0/3935412590 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04105ef0 0x7fed0410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 --2- 172.21.15.103:0/3935412590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 0x7fed041059b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 --2- 172.21.15.103:0/3935412590 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed04101220 0x7fed04101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 -- 172.21.15.103:0/3935412590 >> 172.21.15.103:0/3935412590 conn(0x7fed04078100 msgr2=0x7fed040ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 -- 172.21.15.103:0/3935412590 shutdown_connections 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.510+0000 7fed0be00640 1 -- 172.21.15.103:0/3935412590 wait complete. 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 Processor -- start 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 -- start start 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04101220 0x7fed0419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 0x7fed0419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed04105ef0 0x7fed041a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed04112db0 con 0x7fed04105570 2024-09-11T00:23:15.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed04112c30 con 0x7fed04105ef0 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.511+0000 7fed0be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed04112f30 con 0x7fed04101220 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed09400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04101220 0x7fed0419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed08a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 0x7fed0419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed09400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04101220 0x7fed0419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39448/0 (socket says 172.21.15.103:39448) 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed09400640 1 -- 172.21.15.103:0/1606538848 learned_addr learned my addr 172.21.15.103:0/1606538848 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed09e00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed04105ef0 0x7fed041a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed08a00640 1 -- 172.21.15.103:0/1606538848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04101220 msgr2=0x7fed0419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed08a00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04101220 0x7fed0419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed08a00640 1 -- 172.21.15.103:0/1606538848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed04105ef0 msgr2=0x7fed041a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed08a00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed04105ef0 0x7fed041a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed08a00640 1 -- 172.21.15.103:0/1606538848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041a1550 con 0x7fed04105570 2024-09-11T00:23:15.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.512+0000 7fed09e00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed04105ef0 0x7fed041a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:15.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.513+0000 7fed08a00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 0x7fed0419b9e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fecf002a930 tx=0x7fecf0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:15.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.513+0000 7fed02000640 1 -- 172.21.15.103:0/1606538848 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf003f030 con 0x7fed04105570 2024-09-11T00:23:15.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.513+0000 7fed02000640 1 -- 172.21.15.103:0/1606538848 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecf0004490 con 0x7fed04105570 2024-09-11T00:23:15.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.513+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041a17e0 con 0x7fed04105570 2024-09-11T00:23:15.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.513+0000 7fed02000640 1 -- 172.21.15.103:0/1606538848 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf0035070 con 0x7fed04105570 2024-09-11T00:23:15.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.513+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041a7f00 con 0x7fed04105570 2024-09-11T00:23:15.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.514+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed04101600 con 0x7fed04105570 2024-09-11T00:23:15.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.516+0000 7fed02000640 1 -- 172.21.15.103:0/1606538848 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fecf004c060 con 0x7fed04105570 2024-09-11T00:23:15.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.518+0000 7fed02000640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fecec0b5360 0x7fecec0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:15.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.518+0000 7fed02000640 1 -- 172.21.15.103:0/1606538848 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fecf0105e00 con 0x7fed04105570 2024-09-11T00:23:15.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.518+0000 7fed09400640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fecec0b5360 0x7fecec0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:15.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.519+0000 7fed09400640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fecec0b5360 0x7fecec0b7820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009fd0 tx=0x7fecf8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:15.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.522+0000 7fed02000640 1 -- 172.21.15.103:0/1606538848 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fecf00cc280 con 0x7fed04105570 2024-09-11T00:23:15.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.706+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed041a8230 con 0x7fed04105570 2024-09-11T00:23:15.710 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.708+0000 7fed02000640 1 -- 172.21.15.103:0/1606538848 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24865 (secure 0 0 0) 0x7fecf00d0cb0 con 0x7fed04105570 2024-09-11T00:23:15.710 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:15.711 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":89,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:22:28.671420+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:23:15.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.710+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fecec0b5360 msgr2=0x7fecec0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:15.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.710+0000 7fed0be00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fecec0b5360 0x7fecec0b7820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009fd0 tx=0x7fecf8009500 comp rx=0 tx=0).stop 2024-09-11T00:23:15.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.710+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 msgr2=0x7fed0419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:15.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.710+0000 7fed0be00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 0x7fed0419b9e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fecf002a930 tx=0x7fecf0031a20 comp rx=0 tx=0).stop 2024-09-11T00:23:15.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 shutdown_connections 2024-09-11T00:23:15.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fecec0b5360 0x7fecec0b7820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed04105ef0 0x7fed041a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed04105570 0x7fed0419b9e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 --2- 172.21.15.103:0/1606538848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed04101220 0x7fed0419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:15.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 >> 172.21.15.103:0/1606538848 conn(0x7fed04078100 msgr2=0x7fed04108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:15.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 shutdown_connections 2024-09-11T00:23:15.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:15.711+0000 7fed0be00640 1 -- 172.21.15.103:0/1606538848 wait complete. 2024-09-11T00:23:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:15 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1606538848' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:23:16.096 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:15 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1606538848' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:23:16.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:15 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1606538848' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:23:16.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:20:28.229329+0000', 'modified': '2024-09-11T00:21:27.247376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24656, 'mds_1': 24643, 'mds_2': 24628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24599': {'gid': 24599, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/2973217634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2973217634}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2973217634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6837/2700270511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 2700270511}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2700270511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24622': {'gid': 24622, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/3946403859', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 3946403859}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3946403859}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24628': {'gid': 24628, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.183:6837/335217865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 335217865}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 335217865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24643': {'gid': 24643, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.183:6839/424027451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 424027451}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 424027451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/3769588373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 3769588373}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 3769588373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24656, 'qdb_cluster': [24656, 24643, 24628]} 2024-09-11T00:23:16.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T00:23:16.614 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T00:23:16.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:16 smithi103 ceph-mon[30578]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:16.988 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:17.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:16 smithi183 ceph-mon[35934]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:17.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:16 smithi117 ceph-mon[35525]: pgmap v404: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 -- 172.21.15.103:0/2102351833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8102c10 msgr2=0x7f71d8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 --2- 172.21.15.103:0/2102351833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8102c10 0x7f71d8102ff0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009900 tx=0x7f71cc02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 -- 172.21.15.103:0/2102351833 shutdown_connections 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 --2- 172.21.15.103:0/2102351833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8108fa0 0x7f71d810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 --2- 172.21.15.103:0/2102351833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108620 0x7f71d8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 --2- 172.21.15.103:0/2102351833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8102c10 0x7f71d8102ff0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 -- 172.21.15.103:0/2102351833 >> 172.21.15.103:0/2102351833 conn(0x7f71d80fe6c0 msgr2=0x7f71d8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 -- 172.21.15.103:0/2102351833 shutdown_connections 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.603+0000 7f71de000640 1 -- 172.21.15.103:0/2102351833 wait complete. 2024-09-11T00:23:17.610 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.604+0000 7f71de000640 1 Processor -- start 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.604+0000 7f71de000640 1 -- start start 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71de000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8102c10 0x7f71d8199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71de000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8108620 0x7f71d8199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71de000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108fa0 0x7f71d819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71de000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8112c60 con 0x7f71d8108fa0 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71de000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8112ae0 con 0x7f71d8108620 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71de000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8112de0 con 0x7f71d8102c10 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108fa0 0x7f71d819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8102c10 0x7f71d8199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108fa0 0x7f71d819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33284/0 (socket says 172.21.15.103:33284) 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8102c10 0x7f71d8199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39474/0 (socket says 172.21.15.103:39474) 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 -- 172.21.15.103:0/2298420969 learned_addr learned my addr 172.21.15.103:0/2298420969 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:17.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71d7e00640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8108620 0x7f71d8199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 -- 172.21.15.103:0/2298420969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8102c10 msgr2=0x7f71d8199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8102c10 0x7f71d8199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 -- 172.21.15.103:0/2298420969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8108620 msgr2=0x7f71d8199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8108620 0x7f71d8199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71dd600640 1 -- 172.21.15.103:0/2298420969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d819f0c0 con 0x7f71d8108fa0 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.605+0000 7f71d7e00640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8108620 0x7f71d8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.606+0000 7f71dd600640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108fa0 0x7f71d819e8f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f71d000eb10 tx=0x7f71d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.606+0000 7f71d5600640 1 -- 172.21.15.103:0/2298420969 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0004b20 con 0x7f71d8108fa0 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.606+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d819f3b0 con 0x7f71d8108fa0 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.606+0000 7f71d5600640 1 -- 172.21.15.103:0/2298420969 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71d0004cc0 con 0x7f71d8108fa0 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.606+0000 7f71d5600640 1 -- 172.21.15.103:0/2298420969 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0010690 con 0x7f71d8108fa0 2024-09-11T00:23:17.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.606+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d8075550 con 0x7f71d8108fa0 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.609+0000 7f71e20d9740 1 Processor -- start 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71d5600640 1 -- 172.21.15.103:0/2298420969 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f71d0005020 con 0x7f71d8108fa0 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71e20d9740 1 -- start start 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdcf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fddb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71e20d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56104fd46d50 con 0x56104fdcf400 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71e20d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56104fd36180 con 0x56104fdcfd20 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71e20d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56104fd39f90 con 0x56104fddbcb0 2024-09-11T00:23:17.614 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.610+0000 7f71d7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdcf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fddb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71d7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdcf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33286/0 (socket says 172.21.15.103:33286) 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71d7e00640 1 -- 172.21.15.103:0/3110473702 learned_addr learned my addr 172.21.15.103:0/3110473702 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71d7e00640 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71dcc00640 1 -- 172.21.15.103:0/3110473702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 msgr2=0x56104fdde0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71dcc00640 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdde0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71dcc00640 1 -- 172.21.15.103:0/3110473702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 msgr2=0x56104fdcf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71dcc00640 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdcf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71dcc00640 1 -- 172.21.15.103:0/3110473702 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104fd00a70 con 0x56104fdcfd20 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.611+0000 7f71dcc00640 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fddb770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009230 tx=0x7f71cc005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc22a2ed1b656f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.612+0000 7f71c9c00640 1 -- 172.21.15.103:0/3110473702 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc035050 con 0x56104fdcfd20 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.612+0000 7f71c9c00640 1 -- 172.21.15.103:0/3110473702 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71cc032da0 con 0x56104fdcfd20 2024-09-11T00:23:17.615 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.612+0000 7f71c9c00640 1 -- 172.21.15.103:0/3110473702 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc005280 con 0x56104fdcfd20 2024-09-11T00:23:17.616 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.612+0000 7f71d5600640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71c00b5410 0x7f71c00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.612+0000 7f71d5600640 1 -- 172.21.15.103:0/2298420969 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f71d00e64c0 con 0x7f71d8108fa0 2024-09-11T00:23:17.618 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.612+0000 7f71d7e00640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71c00b5410 0x7f71c00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 -- 172.21.15.103:0/3110473702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 msgr2=0x56104fddb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fddb770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71cc009230 tx=0x7f71cc005e40 comp rx=0 tx=0).stop 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 -- 172.21.15.103:0/3110473702 shutdown_connections 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdde0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fddb770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3110473702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdcf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 -- 172.21.15.103:0/3110473702 >> 172.21.15.103:0/3110473702 conn(0x56104fdcd230 msgr2=0x56104fdce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 -- 172.21.15.103:0/3110473702 shutdown_connections 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71e20d9740 1 -- 172.21.15.103:0/3110473702 wait complete. 2024-09-11T00:23:17.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.613+0000 7f71d7e00640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71c00b5410 0x7f71c00b78d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56104fdcf3b0 tx=0x7f71c4015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 -- start start 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 Processor -- start 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdd8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56104fb975f0 con 0x56104fddbcb0 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56104fcd1460 con 0x56104fdcfd20 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56104fdde9a0 con 0x56104fdcf400 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.614+0000 7f71e20d9740 10 client.0 _ll_register_callbacks cb 0x56104fce1210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:17.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdd8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdd8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39490/0 (socket says 172.21.15.103:39490) 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 -- 172.21.15.103:0/430384204 learned_addr learned my addr 172.21.15.103:0/430384204 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dd600640 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71d7e00640 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 -- 172.21.15.103:0/430384204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 msgr2=0x56104fdd9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 -- 172.21.15.103:0/430384204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 msgr2=0x56104fdd9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 -- 172.21.15.103:0/430384204 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104fa94540 con 0x56104fdcf400 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71dcc00640 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdd8c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71cc031750 tx=0x7f71cc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eab9a80adab0d67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.615+0000 7f71be000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.616+0000 7f71be000640 1 -- 172.21.15.103:0/430384204 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc052070 con 0x56104fdcf400 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.616+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56104fcd3a70 con 0x56104fdcf400 2024-09-11T00:23:17.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.616+0000 7f71be000640 1 -- 172.21.15.103:0/430384204 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f71cc005450 con 0x56104fdcf400 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.616+0000 7f71be000640 1 -- 172.21.15.103:0/430384204 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71cc035020 con 0x56104fdcf400 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.616+0000 7f71e20d9740 10 client.25240 fetch_fsmap learned FSMap version 26 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.616+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56104fcee6f0 con 0x56104fdcf400 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.616+0000 7f71be000640 1 -- 172.21.15.103:0/430384204 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56104fcd3a70 con 0x56104fdcf400 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71be000640 1 -- 172.21.15.103:0/430384204 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f71cc03a700 con 0x56104fdcf400 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71e20d9740 10 client.25240 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71e20d9740 10 client.25240 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71e20d9740 20 client.25240 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71e20d9740 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fdec530 0x56104fded9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71e20d9740 4 client.25240 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:17.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 --> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104fd48ca0 con 0x56104fdec530 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.617+0000 7f71d7e00640 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fdec530 0x56104fded9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.618+0000 7f71be000640 10 client.25240 ms_handle_connect on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.618+0000 7f71d7e00640 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fdec530 0x56104fded9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78b2fe837bfca090 server_cookie=2b89d3b99d2fd014 in_seq=0 out_seq=0 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71be000640 1 -- 172.21.15.103:0/430384204 <== mds.2 v2:172.21.15.183:6836/335217865 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56104fd48ca0 con 0x56104fdec530 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71be000640 10 client.25240 handle_command_reply: tid=0 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 client.25240 shutdown 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 msgr2=0x56104fdd8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdd8c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71cc031750 tx=0x7f71cc009260 comp rx=0 tx=0).stop 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 shutdown_connections 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fdec530 0x56104fded9a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 --2- 172.21.15.103:0/430384204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdd8c10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71be000640 0 client.25240 ms_handle_reset on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:17.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 >> 172.21.15.103:0/430384204 conn(0x56104fdcd230 msgr2=0x56104fdd2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:17.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 shutdown_connections 2024-09-11T00:23:17.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 1 -- 172.21.15.103:0/430384204 wait complete. 2024-09-11T00:23:17.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.619+0000 7f71e20d9740 20 client.25240 trim_cache size 0 max 16384 2024-09-11T00:23:17.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 Processor -- start 2024-09-11T00:23:17.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 -- start start 2024-09-11T00:23:17.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdeb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.640 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56104fdebce0 con 0x56104fddbcb0 2024-09-11T00:23:17.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56104fdec3b0 con 0x56104fdcfd20 2024-09-11T00:23:17.641 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.638+0000 7f71e20d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56104fdeca80 con 0x56104fdcf400 2024-09-11T00:23:17.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71dd600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:35636/0 (socket says 172.21.15.103:35636) 2024-09-11T00:23:17.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 -- 172.21.15.103:0/3907162606 learned_addr learned my addr 172.21.15.103:0/3907162606 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:17.642 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71dcc00640 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdeb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 -- 172.21.15.103:0/3907162606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 msgr2=0x56104fdeb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdeb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 -- 172.21.15.103:0/3907162606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 msgr2=0x56104fdd5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 -- 172.21.15.103:0/3907162606 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104fdcdcb0 con 0x56104fdcfd20 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71dcc00640 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdeb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71d7e00640 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd3470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f71c40136c0 tx=0x7f71c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7226930b3c672129 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.639+0000 7f71c9200640 1 -- 172.21.15.103:0/3907162606 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4030020 con 0x56104fdcfd20 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71c9200640 1 -- 172.21.15.103:0/3907162606 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4022a40 con 0x56104fdcfd20 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71c9200640 1 -- 172.21.15.103:0/3907162606 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4021a50 con 0x56104fdcfd20 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 -- 172.21.15.103:0/3907162606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 msgr2=0x56104fdd3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd3470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f71c40136c0 tx=0x7f71c40046c0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 -- 172.21.15.103:0/3907162606 shutdown_connections 2024-09-11T00:23:17.643 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fddbcb0 0x56104fdd5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd3470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 --2- 172.21.15.103:0/3907162606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fdcf400 0x56104fdeb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 -- 172.21.15.103:0/3907162606 >> 172.21.15.103:0/3907162606 conn(0x56104fdcd230 msgr2=0x56104fdd3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:17.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 -- 172.21.15.103:0/3907162606 shutdown_connections 2024-09-11T00:23:17.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.640+0000 7f71e20d9740 1 -- 172.21.15.103:0/3907162606 wait complete. 2024-09-11T00:23:17.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.641+0000 7f71e20d9740 1 -- start start 2024-09-11T00:23:17.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.641+0000 7f71e20d9740 1 Processor -- start 2024-09-11T00:23:17.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.641+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdf6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71e20d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdd6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71e20d9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56104fded820 con 0x56104fdcf400 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71e20d9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56104fded150 con 0x56104fdcfd20 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71e20d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56104fdedef0 con 0x56104fddbcb0 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71e20d9740 10 client.0 _ll_register_callbacks cb 0x7f71cc032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdf6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdf6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33310/0 (socket says 172.21.15.103:33310) 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 -- 172.21.15.103:0/1519885422 learned_addr learned my addr 172.21.15.103:0/1519885422 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71d7e00640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dd600640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdd6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 -- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 msgr2=0x56104fdd6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdd6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 -- 172.21.15.103:0/1519885422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 msgr2=0x56104fdd5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dcc00640 1 -- 172.21.15.103:0/1519885422 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104fdf52a0 con 0x56104fdcf400 2024-09-11T00:23:17.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71dd600640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdd6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.642+0000 7f71d7e00640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71dcc00640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdf6640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71cc005ec0 tx=0x7f71cc0329c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27632a6ea1617b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71be000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71be000640 1 -- 172.21.15.103:0/1519885422 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71cc052070 con 0x56104fdcf400 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71be000640 1 -- 172.21.15.103:0/1519885422 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f71cc005250 con 0x56104fdcf400 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71c4022a40 con 0x56104fdcf400 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71be000640 1 -- 172.21.15.103:0/1519885422 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f71cc069050 con 0x56104fdcf400 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71e20d9740 10 client.15210 fetch_fsmap learned FSMap version 26 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56104fe59110 con 0x56104fdcf400 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.643+0000 7f71be000640 1 -- 172.21.15.103:0/1519885422 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71c4022a40 con 0x56104fdcf400 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71be000640 1 -- 172.21.15.103:0/1519885422 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f71cc04e440 con 0x56104fdcf400 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71e20d9740 10 client.15210 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:23:17.648 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71e20d9740 10 client.15210 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:23:17.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71e20d9740 20 client.15210 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:17.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71e20d9740 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fe5dce0 0x56104fe5c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:17.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71e20d9740 4 client.15210 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:23:17.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 --> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] -- command(tid 0: {"prefix": "status"}) -- 0x56104fd44750 con 0x56104fe5dce0 2024-09-11T00:23:17.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.644+0000 7f71d7e00640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fe5dce0 0x56104fe5c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71d7e00640 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fe5dce0 0x56104fe5c9f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9aaf5c0172c7c725 server_cookie=ad4d1fb50f878846 in_seq=0 out_seq=0 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71be000640 10 client.15210 ms_handle_connect on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71be000640 1 -- 172.21.15.103:0/1519885422 <== mds.2 v2:172.21.15.183:6836/335217865 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56104fd44750 con 0x56104fe5dce0 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71be000640 10 client.15210 handle_command_reply: tid=0 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 client.15210 shutdown 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 msgr2=0x56104fdf6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdf6640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f71cc005ec0 tx=0x7f71cc0329c0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.650 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 shutdown_connections 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x56104fe5dce0 0x56104fe5c9f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56104fddbcb0 0x56104fdd6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56104fdcfd20 0x56104fdd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 --2- 172.21.15.103:0/1519885422 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56104fdcf400 0x56104fdf6640 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 >> 172.21.15.103:0/1519885422 conn(0x56104fdcd230 msgr2=0x56104fde6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.645+0000 7f71be000640 0 client.15210 ms_handle_reset on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.646+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 shutdown_connections 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.646+0000 7f71e20d9740 1 -- 172.21.15.103:0/1519885422 wait complete. 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.646+0000 7f71e20d9740 20 client.15210 trim_cache size 0 max 16384 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:23:17.651 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24628, 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 114.526070127, 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 200.314048926 2024-09-11T00:23:17.652 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.646+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71c00b5410 msgr2=0x7f71c00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.646+0000 7f71de000640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71c00b5410 0x7f71c00b78d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56104fdcf3b0 tx=0x7f71c4015040 comp rx=0 tx=0).stop 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.646+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108fa0 msgr2=0x7f71d819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.646+0000 7f71de000640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108fa0 0x7f71d819e8f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f71d000eb10 tx=0x7f71d000c2d0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 shutdown_connections 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f71c00b5410 0x7f71c00b78d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71d8108fa0 0x7f71d819e8f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d8108620 0x7f71d8199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 --2- 172.21.15.103:0/2298420969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71d8102c10 0x7f71d8199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:17.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 >> 172.21.15.103:0/2298420969 conn(0x7f71d80fe6c0 msgr2=0x7f71d8100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:17.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 shutdown_connections 2024-09-11T00:23:17.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:17.647+0000 7f71de000640 1 -- 172.21.15.103:0/2298420969 wait complete. 2024-09-11T00:23:18.655 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:23:18.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:18 smithi183 ceph-mon[35934]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:23:18.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:18 smithi103 ceph-mon[30578]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:23:18.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:18 smithi117 ceph-mon[35525]: pgmap v405: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:23:19.028 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:19.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.702+0000 7f8e17e00640 1 -- 172.21.15.103:0/1012093216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 msgr2=0x7f8e181119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.702+0000 7f8e17e00640 1 --2- 172.21.15.103:0/1012093216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 0x7f8e181119a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009ca0 tx=0x7f8e1002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.702+0000 7f8e17e00640 1 -- 172.21.15.103:0/1012093216 shutdown_connections 2024-09-11T00:23:19.712 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.702+0000 7f8e17e00640 1 --2- 172.21.15.103:0/1012093216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 0x7f8e181119a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.702+0000 7f8e17e00640 1 --2- 172.21.15.103:0/1012093216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e1810a820 0x7f8e1810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.702+0000 7f8e17e00640 1 --2- 172.21.15.103:0/1012093216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e18104e10 0x7f8e181051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.702+0000 7f8e17e00640 1 -- 172.21.15.103:0/1012093216 >> 172.21.15.103:0/1012093216 conn(0x7f8e18100840 msgr2=0x7f8e18102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.703+0000 7f8e17e00640 1 -- 172.21.15.103:0/1012093216 shutdown_connections 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.703+0000 7f8e17e00640 1 -- 172.21.15.103:0/1012093216 wait complete. 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 Processor -- start 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 -- start start 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e18104e10 0x7f8e1819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e1810a820 0x7f8e1819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 0x7f8e181a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18114e60 con 0x7f8e1810b1a0 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18114ce0 con 0x7f8e18104e10 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.704+0000 7f8e17e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18114fe0 con 0x7f8e1810a820 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e18104e10 0x7f8e1819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e18104e10 0x7f8e1819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:35666/0 (socket says 172.21.15.103:35666) 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e16a00640 1 -- 172.21.15.103:0/644827916 learned_addr learned my addr 172.21.15.103:0/644827916 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e17400640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 0x7f8e181a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e16000640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e1810a820 0x7f8e1819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e17400640 1 -- 172.21.15.103:0/644827916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e1810a820 msgr2=0x7f8e1819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e17400640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e1810a820 0x7f8e1819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e17400640 1 -- 172.21.15.103:0/644827916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e18104e10 msgr2=0x7f8e1819b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e17400640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e18104e10 0x7f8e1819b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.714 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e17400640 1 -- 172.21.15.103:0/644827916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e181a1260 con 0x7f8e1810b1a0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e16a00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e18104e10 0x7f8e1819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.705+0000 7f8e17400640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 0x7f8e181a0a90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e1002ade0 tx=0x7f8e10031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.706+0000 7f8e1ca00640 1 -- 172.21.15.103:0/644827916 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10003b70 con 0x7f8e1810b1a0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.706+0000 7f8e1ca00640 1 -- 172.21.15.103:0/644827916 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e10031910 con 0x7f8e1810b1a0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.706+0000 7f8e1ca00640 1 -- 172.21.15.103:0/644827916 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1002f950 con 0x7f8e1810b1a0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.706+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e181a14f0 con 0x7f8e1810b1a0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.706+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e181a7c10 con 0x7f8e1810b1a0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.710+0000 7f8e20a35740 1 Processor -- start 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.710+0000 7f8e1ca00640 1 -- 172.21.15.103:0/644827916 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8e1002faf0 con 0x7f8e1810b1a0 2024-09-11T00:23:19.715 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.710+0000 7f8e20a35740 1 -- start start 2024-09-11T00:23:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0d37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3d20 0x55928e0df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0e20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e20a35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55928e04ad50 con 0x55928e0dfcb0 2024-09-11T00:23:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e20a35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55928e03a180 con 0x55928e0d3400 2024-09-11T00:23:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e20a35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55928e03df90 con 0x55928e0d3d20 2024-09-11T00:23:19.719 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e16000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0d37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3d20 0x55928e0df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3d20 0x55928e0df770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39534/0 (socket says 172.21.15.103:39534) 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e16000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0e20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.711+0000 7f8e16a00640 1 -- 172.21.15.103:0/784672126 learned_addr learned my addr 172.21.15.103:0/784672126 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e16a00640 1 -- 172.21.15.103:0/784672126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 msgr2=0x55928e0d37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e16a00640 1 --2- 172.21.15.103:0/784672126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0d37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e16a00640 1 -- 172.21.15.103:0/784672126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 msgr2=0x55928e0e20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e16a00640 1 --2- 172.21.15.103:0/784672126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0e20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e16a00640 1 -- 172.21.15.103:0/784672126 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928e004a70 con 0x55928e0d3d20 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e16a00640 1 --2- 172.21.15.103:0/784672126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3d20 0x55928e0df770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e08011660 tx=0x7f8e08011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602fefc0558c3868 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e1ca00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dfc0b5410 0x7f8dfc0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.720 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e03e00640 1 -- 172.21.15.103:0/784672126 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0801e070 con 0x55928e0d3d20 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e03e00640 1 -- 172.21.15.103:0/784672126 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e0800fba0 con 0x55928e0d3d20 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e03e00640 1 -- 172.21.15.103:0/784672126 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08019e60 con 0x55928e0d3d20 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e1ca00640 1 -- 172.21.15.103:0/644827916 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8e10106fa0 con 0x7f8e1810b1a0 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.712+0000 7f8e16000640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dfc0b5410 0x7f8dfc0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 -- 172.21.15.103:0/784672126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3d20 msgr2=0x55928e0df770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e16000640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dfc0b5410 0x7f8dfc0b78d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55928e0d33b0 tx=0x7f8e04004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 --2- 172.21.15.103:0/784672126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3d20 0x55928e0df770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8e08011660 tx=0x7f8e08011b20 comp rx=0 tx=0).stop 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 -- 172.21.15.103:0/784672126 shutdown_connections 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 --2- 172.21.15.103:0/784672126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0e20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 --2- 172.21.15.103:0/784672126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3d20 0x55928e0df770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 --2- 172.21.15.103:0/784672126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0d37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 -- 172.21.15.103:0/784672126 >> 172.21.15.103:0/784672126 conn(0x55928e0d1230 msgr2=0x55928e0d23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:19.721 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 -- 172.21.15.103:0/784672126 shutdown_connections 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.713+0000 7f8e20a35740 1 -- 172.21.15.103:0/784672126 wait complete. 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.714+0000 7f8e20a35740 1 -- start start 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.714+0000 7f8e20a35740 1 Processor -- start 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0dcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0d3d20 0x55928e0dd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0dfcb0 0x55928e0dd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e20a35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55928de9b5f0 con 0x55928e0d3d20 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e20a35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55928dfd5460 con 0x55928e0d3400 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e20a35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55928e0e2a00 con 0x55928e0dfcb0 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e20a35740 10 client.0 _ll_register_callbacks cb 0x55928dfe5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0dcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0d3d20 0x55928e0dd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e17400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0dfcb0 0x55928e0dd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0dcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:35696/0 (socket says 172.21.15.103:35696) 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 -- 172.21.15.103:0/680485480 learned_addr learned my addr 172.21.15.103:0/680485480 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 -- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0dfcb0 msgr2=0x55928e0dd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0dfcb0 0x55928e0dd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 -- 172.21.15.103:0/680485480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0d3d20 msgr2=0x55928e0dd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0d3d20 0x55928e0dd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e16a00640 1 -- 172.21.15.103:0/680485480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928dd98540 con 0x55928e0d3400 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.715+0000 7f8e17400640 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0dfcb0 0x55928e0dd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e16a00640 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0dcc50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e08011630 tx=0x7f8e08009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13d94083e8fe6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:19.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e00c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e00c00640 1 -- 172.21.15.103:0/680485480 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0801e050 con 0x55928e0d3400 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55928dfd7a70 con 0x55928e0d3400 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e00c00640 1 -- 172.21.15.103:0/680485480 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8e0800fdb0 con 0x55928e0d3400 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e00c00640 1 -- 172.21.15.103:0/680485480 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e08033050 con 0x55928e0d3400 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e20a35740 10 client.25232 fetch_fsmap learned FSMap version 26 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.716+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55928dff26f0 con 0x55928e0d3400 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e00c00640 1 -- 172.21.15.103:0/680485480 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55928dfd7a70 con 0x55928e0d3400 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e00c00640 1 -- 172.21.15.103:0/680485480 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e08004900 con 0x55928e0d3400 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e20a35740 10 client.25232 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e20a35740 10 client.25232 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e20a35740 20 client.25232 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:19.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e20a35740 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e0f0850 0x55928e0f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e20a35740 4 client.25232 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:19.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.717+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 --> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928e04cca0 con 0x55928e0f0850 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.718+0000 7f8e16000640 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e0f0850 0x55928e0f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.718+0000 7f8e16000640 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e0f0850 0x55928e0f1cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7000a1ecb344766 server_cookie=68a713103f0e6637 in_seq=0 out_seq=0 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.718+0000 7f8e00c00640 10 client.25232 ms_handle_connect on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e00c00640 1 -- 172.21.15.103:0/680485480 <== mds.1 v2:172.21.15.183:6838/424027451 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55928e04cca0 con 0x55928e0f0850 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e00c00640 10 client.25232 handle_command_reply: tid=0 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 client.25232 shutdown 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 msgr2=0x55928e0dcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0dcc50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8e08011630 tx=0x7f8e08009f10 comp rx=0 tx=0).stop 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 shutdown_connections 2024-09-11T00:23:19.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e0f0850 0x55928e0f1cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0dfcb0 0x55928e0dd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0d3d20 0x55928e0dd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e00c00640 0 client.25232 ms_handle_reset on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 --2- 172.21.15.103:0/680485480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3400 0x55928e0dcc50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 >> 172.21.15.103:0/680485480 conn(0x55928e0d1230 msgr2=0x55928e0d6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 shutdown_connections 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.719+0000 7f8e20a35740 1 -- 172.21.15.103:0/680485480 wait complete. 2024-09-11T00:23:19.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.720+0000 7f8e20a35740 20 client.25232 trim_cache size 0 max 16384 2024-09-11T00:23:19.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.738+0000 7f8e20a35740 1 Processor -- start 2024-09-11T00:23:19.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.738+0000 7f8e20a35740 1 -- start start 2024-09-11T00:23:19.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.738+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0d7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.740 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.738+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.738+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.738+0000 7f8e20a35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55928e0f0000 con 0x55928e0dfcb0 2024-09-11T00:23:19.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.738+0000 7f8e20a35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55928e0f06d0 con 0x55928e0d3d20 2024-09-11T00:23:19.741 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e20a35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55928e0f0da0 con 0x55928e0d3400 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e17400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0db4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:35698/0 (socket says 172.21.15.103:35698) 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 -- 172.21.15.103:0/3406800763 learned_addr learned my addr 172.21.15.103:0/3406800763 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16a00640 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0d7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 -- 172.21.15.103:0/3406800763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 msgr2=0x55928e0d7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0d7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 -- 172.21.15.103:0/3406800763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 msgr2=0x55928e0dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.742 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 -- 172.21.15.103:0/3406800763 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928e0e24e0 con 0x55928e0d3d20 2024-09-11T00:23:19.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e17400640 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:19.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.739+0000 7f8e16000640 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0db4d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55928e0f2520 tx=0x7f8e04012070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6016f52a86194898 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:19.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e03400640 1 -- 172.21.15.103:0/3406800763 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04019040 con 0x55928e0d3d20 2024-09-11T00:23:19.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e03400640 1 -- 172.21.15.103:0/3406800763 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e04010d90 con 0x55928e0d3d20 2024-09-11T00:23:19.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e03400640 1 -- 172.21.15.103:0/3406800763 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04005580 con 0x55928e0d3d20 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e20a35740 1 -- 172.21.15.103:0/3406800763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 msgr2=0x55928e0db4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e20a35740 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0db4d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55928e0f2520 tx=0x7f8e04012070 comp rx=0 tx=0).stop 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e20a35740 1 -- 172.21.15.103:0/3406800763 shutdown_connections 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e20a35740 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.740+0000 7f8e20a35740 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0db4d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.741+0000 7f8e20a35740 1 --2- 172.21.15.103:0/3406800763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0d7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.741+0000 7f8e20a35740 1 -- 172.21.15.103:0/3406800763 >> 172.21.15.103:0/3406800763 conn(0x55928e0d1230 msgr2=0x55928e0d7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.741+0000 7f8e20a35740 1 -- 172.21.15.103:0/3406800763 shutdown_connections 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.741+0000 7f8e20a35740 1 -- 172.21.15.103:0/3406800763 wait complete. 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.741+0000 7f8e20a35740 1 -- start start 2024-09-11T00:23:19.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.741+0000 7f8e20a35740 1 Processor -- start 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0fa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e20a35740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e20a35740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55928e0f1b40 con 0x55928e0dfcb0 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e20a35740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55928e0f1470 con 0x55928e0d3d20 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e20a35740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55928e0f2210 con 0x55928e0d3400 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e20a35740 10 client.0 _ll_register_callbacks cb 0x7f8e04009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e17400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39580/0 (socket says 172.21.15.103:39580) 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 -- 172.21.15.103:0/2832270086 learned_addr learned my addr 172.21.15.103:0/2832270086 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:19.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16000640 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 -- 172.21.15.103:0/2832270086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 msgr2=0x55928e0fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 -- 172.21.15.103:0/2832270086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 msgr2=0x55928e0f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.742+0000 7f8e16a00640 1 -- 172.21.15.103:0/2832270086 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928e0f3f30 con 0x55928e0d3400 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e16a00640 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0fa190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e08004ec0 tx=0x7f8e0800fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a48fa69213c59231 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e00c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e00c00640 1 -- 172.21.15.103:0/2832270086 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0803b070 con 0x55928e0d3400 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e04010d90 con 0x55928e0d3400 2024-09-11T00:23:19.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e00c00640 1 -- 172.21.15.103:0/2832270086 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8e08004680 con 0x55928e0d3400 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e00c00640 1 -- 172.21.15.103:0/2832270086 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8e08047050 con 0x55928e0d3400 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e20a35740 10 client.25264 fetch_fsmap learned FSMap version 26 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.743+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55928e0f98d0 con 0x55928e0d3400 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e00c00640 1 -- 172.21.15.103:0/2832270086 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e04010d90 con 0x55928e0d3400 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e00c00640 1 -- 172.21.15.103:0/2832270086 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8e080054c0 con 0x55928e0d3400 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e20a35740 10 client.25264 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e20a35740 10 client.25264 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e20a35740 20 client.25264 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e20a35740 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e16a000 0x55928e168d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e20a35740 4 client.25264 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:23:19.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 --> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] -- command(tid 0: {"prefix": "status"}) -- 0x55928e048750 con 0x55928e16a000 2024-09-11T00:23:19.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.744+0000 7f8e16000640 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e16a000 0x55928e168d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:19.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e16000640 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e16a000 0x55928e168d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5e234ed147bff63 server_cookie=17bc932b5bd22713 in_seq=0 out_seq=0 2024-09-11T00:23:19.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e00c00640 10 client.25264 ms_handle_connect on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e00c00640 1 -- 172.21.15.103:0/2832270086 <== mds.1 v2:172.21.15.183:6838/424027451 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55928e048750 con 0x55928e16a000 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e00c00640 10 client.25264 handle_command_reply: tid=0 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e20a35740 1 client.25264 shutdown 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 msgr2=0x55928e0fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e20a35740 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0fa190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e08004ec0 tx=0x7f8e0800fdc0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 shutdown_connections 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e20a35740 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x55928e16a000 0x55928e168d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e20a35740 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55928e0dfcb0 0x55928e0f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e00c00640 0 client.25264 ms_handle_reset on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.745+0000 7f8e20a35740 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55928e0d3d20 0x55928e0fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.746+0000 7f8e20a35740 1 --2- 172.21.15.103:0/2832270086 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55928e0d3400 0x55928e0fa190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.746+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 >> 172.21.15.103:0/2832270086 conn(0x55928e0d1230 msgr2=0x55928e0d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.746+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 shutdown_connections 2024-09-11T00:23:19.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.746+0000 7f8e20a35740 1 -- 172.21.15.103:0/2832270086 wait complete. 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.746+0000 7f8e20a35740 20 client.25264 trim_cache size 0 max 16384 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24643, 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 117.640327743, 2024-09-11T00:23:19.755 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-11T00:23:19.756 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:23:19.756 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T00:23:19.756 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 197.07552061999999 2024-09-11T00:23:19.756 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:23:19.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dfc0b5410 msgr2=0x7f8dfc0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dfc0b5410 0x7f8dfc0b78d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55928e0d33b0 tx=0x7f8e04004730 comp rx=0 tx=0).stop 2024-09-11T00:23:19.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 msgr2=0x7f8e181a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 0x7f8e181a0a90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8e1002ade0 tx=0x7f8e10031660 comp rx=0 tx=0).stop 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 shutdown_connections 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8dfc0b5410 0x7f8dfc0b78d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e1810b1a0 0x7f8e181a0a90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e1810a820 0x7f8e1819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 --2- 172.21.15.103:0/644827916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e18104e10 0x7f8e1819b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 >> 172.21.15.103:0/644827916 conn(0x7f8e18100840 msgr2=0x7f8e18103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 shutdown_connections 2024-09-11T00:23:19.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:19.747+0000 7f8e17e00640 1 -- 172.21.15.103:0/644827916 wait complete. 2024-09-11T00:23:20.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:20 smithi103 ceph-mon[30578]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:20 smithi183 ceph-mon[35934]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:20.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:23:20.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:20 smithi117 ceph-mon[35525]: pgmap v406: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:21.170 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 -- 172.21.15.103:0/1915308020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f01087d0 msgr2=0x7f86f0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 --2- 172.21.15.103:0/1915308020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f01087d0 0x7f86f0108c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009900 tx=0x7f86d802a410 comp rx=0 tx=0).stop 2024-09-11T00:23:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 -- 172.21.15.103:0/1915308020 shutdown_connections 2024-09-11T00:23:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 --2- 172.21.15.103:0/1915308020 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f0109150 0x7f86f010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 --2- 172.21.15.103:0/1915308020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f01087d0 0x7f86f0108c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 --2- 172.21.15.103:0/1915308020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f0102dc0 0x7f86f01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 -- 172.21.15.103:0/1915308020 >> 172.21.15.103:0/1915308020 conn(0x7f86f00fe830 msgr2=0x7f86f0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 -- 172.21.15.103:0/1915308020 shutdown_connections 2024-09-11T00:23:21.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.807+0000 7f86f6400640 1 -- 172.21.15.103:0/1915308020 wait complete. 2024-09-11T00:23:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 Processor -- start 2024-09-11T00:23:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 -- start start 2024-09-11T00:23:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f0102dc0 0x7f86f019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f01087d0 0x7f86f019b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f0109150 0x7f86f01a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112e20 con 0x7f86f0109150 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112ca0 con 0x7f86f0102dc0 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86f6400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112fa0 con 0x7f86f01087d0 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f0109150 0x7f86f01a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.808+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f01087d0 0x7f86f019b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f01087d0 0x7f86f019b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39592/0 (socket says 172.21.15.103:39592) 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 -- 172.21.15.103:0/4292342918 learned_addr learned my addr 172.21.15.103:0/4292342918 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86ef400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f0102dc0 0x7f86f019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 -- 172.21.15.103:0/4292342918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f0102dc0 msgr2=0x7f86f019b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f0102dc0 0x7f86f019b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 -- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f0109150 msgr2=0x7f86f01a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f0109150 0x7f86f01a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 -- 172.21.15.103:0/4292342918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f01a1530 con 0x7f86f01087d0 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86ef400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f0102dc0 0x7f86f019b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:21.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.809+0000 7f86e6a00640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f01087d0 0x7f86f019b9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005ab0 tx=0x7f86d8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.810+0000 7f86ecc00640 1 -- 172.21.15.103:0/4292342918 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d803f070 con 0x7f86f01087d0 2024-09-11T00:23:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.810+0000 7f86ecc00640 1 -- 172.21.15.103:0/4292342918 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d8031b60 con 0x7f86f01087d0 2024-09-11T00:23:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.810+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f01a17c0 con 0x7f86f01087d0 2024-09-11T00:23:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.810+0000 7f86ecc00640 1 -- 172.21.15.103:0/4292342918 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8035030 con 0x7f86f01087d0 2024-09-11T00:23:21.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.810+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a7ee0 con 0x7f86f01087d0 2024-09-11T00:23:21.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 Processor -- start 2024-09-11T00:23:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 -- start start 2024-09-11T00:23:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb62620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62b60 0x56369fb6e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb6eaf0 0x56369fb70ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56369fad9d50 con 0x56369fb62240 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56369fac9180 con 0x56369fb6eaf0 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86f9018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56369faccf90 con 0x56369fb62b60 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.813+0000 7f86ecc00640 1 -- 172.21.15.103:0/4292342918 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f86d803b460 con 0x7f86f01087d0 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb62620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb62620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33392/0 (socket says 172.21.15.103:33392) 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 -- 172.21.15.103:0/1998848799 learned_addr learned my addr 172.21.15.103:0/1998848799 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86e6a00640 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb6eaf0 0x56369fb70ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86ef400640 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62b60 0x56369fb6e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 -- 172.21.15.103:0/1998848799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62b60 msgr2=0x56369fb6e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62b60 0x56369fb6e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 -- 172.21.15.103:0/1998848799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb6eaf0 msgr2=0x56369fb70ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb6eaf0 0x56369fb70ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 -- 172.21.15.103:0/1998848799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369fa93a70 con 0x56369fb62240 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.814+0000 7f86efe00640 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb62620 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009cd0 tx=0x7f86e8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4f630d02a40227 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.815+0000 7f86e4c00640 1 -- 172.21.15.103:0/1998848799 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e801e070 con 0x56369fb62240 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.815+0000 7f86e4c00640 1 -- 172.21.15.103:0/1998848799 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e8019ae0 con 0x56369fb62240 2024-09-11T00:23:21.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.815+0000 7f86e4c00640 1 -- 172.21.15.103:0/1998848799 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e800ea20 con 0x56369fb62240 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.815+0000 7f86ecc00640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86d40b52e0 0x7f86d40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.815+0000 7f86ecc00640 1 -- 172.21.15.103:0/4292342918 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f86d8106910 con 0x7f86f01087d0 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.815+0000 7f86ef400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86d40b52e0 0x7f86d40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 -- 172.21.15.103:0/1998848799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 msgr2=0x56369fb62620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb62620 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009cd0 tx=0x7f86e8011810 comp rx=0 tx=0).stop 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 -- 172.21.15.103:0/1998848799 shutdown_connections 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb6eaf0 0x56369fb70ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62b60 0x56369fb6e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 --2- 172.21.15.103:0/1998848799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb62620 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 -- 172.21.15.103:0/1998848799 >> 172.21.15.103:0/1998848799 conn(0x56369fb60020 msgr2=0x56369fb61210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 -- 172.21.15.103:0/1998848799 shutdown_connections 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86f9018740 1 -- 172.21.15.103:0/1998848799 wait complete. 2024-09-11T00:23:21.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.816+0000 7f86ef400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86d40b52e0 0x7f86d40b77a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56369fb653c0 tx=0x7f86e0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 -- start start 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 Processor -- start 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb6ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56369f92a5f0 con 0x56369fb62240 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56369fa64460 con 0x56369fb62b60 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56369fb71840 con 0x56369fb6eaf0 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.817+0000 7f86f9018740 10 client.0 _ll_register_callbacks cb 0x56369fa74210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:21.822 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86ef400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb6ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb6bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:35758/0 (socket says 172.21.15.103:35758) 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86e6a00640 1 -- 172.21.15.103:0/593434703 learned_addr learned my addr 172.21.15.103:0/593434703 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86efe00640 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86ef400640 1 -- 172.21.15.103:0/593434703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 msgr2=0x56369fb6c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86ef400640 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86ef400640 1 -- 172.21.15.103:0/593434703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 msgr2=0x56369fb6bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86ef400640 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb6bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86ef400640 1 -- 172.21.15.103:0/593434703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369f827540 con 0x56369fb62240 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86ef400640 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb6ba90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f86e0005fd0 tx=0x7f86e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70712c3fea5d4615 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.818+0000 7f86dd600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.819+0000 7f86dd600640 1 -- 172.21.15.103:0/593434703 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0011c90 con 0x56369fb62240 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.819+0000 7f86dd600640 1 -- 172.21.15.103:0/593434703 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f86e0003970 con 0x56369fb62240 2024-09-11T00:23:21.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.819+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56369fa66a70 con 0x56369fb62240 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.819+0000 7f86dd600640 1 -- 172.21.15.103:0/593434703 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86e0011e30 con 0x56369fb62240 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.819+0000 7f86f9018740 10 client.15240 fetch_fsmap learned FSMap version 26 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.819+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56369fa816f0 con 0x56369fb62240 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.819+0000 7f86dd600640 1 -- 172.21.15.103:0/593434703 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56369fa66a70 con 0x56369fb62240 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86dd600640 1 -- 172.21.15.103:0/593434703 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f86e0027020 con 0x56369fb62240 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86f9018740 10 client.15240 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86f9018740 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86f9018740 20 client.15240 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86f9018740 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fb7f8a0 0x56369fb80d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86f9018740 4 client.15240 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56369fadbca0 con 0x56369fb7f8a0 2024-09-11T00:23:21.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.820+0000 7f86e6a00640 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fb7f8a0 0x56369fb80d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.825 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.821+0000 7f86e6a00640 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fb7f8a0 0x56369fb80d10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef9040a410da07e server_cookie=c6c8a415a1f9cb1b in_seq=0 out_seq=0 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.821+0000 7f86dd600640 10 client.15240 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.821+0000 7f86dd600640 1 -- 172.21.15.103:0/593434703 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56369fadbca0 con 0x56369fb7f8a0 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.821+0000 7f86dd600640 10 client.15240 handle_command_reply: tid=0 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.821+0000 7f86f9018740 1 client.15240 shutdown 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 msgr2=0x56369fb6ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb6ba90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f86e0005fd0 tx=0x7f86e0004ab0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 shutdown_connections 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fb7f8a0 0x56369fb80d10 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb6bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 --2- 172.21.15.103:0/593434703 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62240 0x56369fb6ba90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 >> 172.21.15.103:0/593434703 conn(0x56369fb60020 msgr2=0x56369fb65c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86dd600640 0 client.15240 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:21.826 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 shutdown_connections 2024-09-11T00:23:21.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 1 -- 172.21.15.103:0/593434703 wait complete. 2024-09-11T00:23:21.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.822+0000 7f86f9018740 20 client.15240 trim_cache size 0 max 16384 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.840+0000 7f86f9018740 1 Processor -- start 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.840+0000 7f86f9018740 1 -- start start 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86f9018740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62240 0x56369fb7f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86f9018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62b60 0x56369fb66400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86f9018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86f9018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56369fb7f050 con 0x56369fb62b60 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86f9018740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56369fb7f720 con 0x56369fb62240 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86f9018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56369fb7fdf0 con 0x56369fb6eaf0 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62b60 0x56369fb66400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86efe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62b60 0x56369fb66400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33414/0 (socket says 172.21.15.103:33414) 2024-09-11T00:23:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 -- 172.21.15.103:0/1322354888 learned_addr learned my addr 172.21.15.103:0/1322354888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:21.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86ef400640 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62240 0x56369fb7f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 -- 172.21.15.103:0/1322354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 msgr2=0x56369fb6a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 -- 172.21.15.103:0/1322354888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62240 msgr2=0x56369fb7f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62240 0x56369fb7f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.845 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.841+0000 7f86e6a00640 1 -- 172.21.15.103:0/1322354888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369fb62950 con 0x56369fb62b60 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.842+0000 7f86e6a00640 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62b60 0x56369fb66400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000c00 tx=0x7f86d8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88f0e468daafe61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.842+0000 7f86dfe00640 1 -- 172.21.15.103:0/1322354888 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8049040 con 0x56369fb62b60 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.842+0000 7f86dfe00640 1 -- 172.21.15.103:0/1322354888 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d80cc8a0 con 0x56369fb62b60 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.842+0000 7f86dfe00640 1 -- 172.21.15.103:0/1322354888 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d80b40d0 con 0x56369fb62b60 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.842+0000 7f86f9018740 1 -- 172.21.15.103:0/1322354888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62b60 msgr2=0x56369fb66400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.842+0000 7f86f9018740 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62b60 0x56369fb66400 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f86d8000c00 tx=0x7f86d8009680 comp rx=0 tx=0).stop 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 -- 172.21.15.103:0/1322354888 shutdown_connections 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb6eaf0 0x56369fb6a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb62b60 0x56369fb66400 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 --2- 172.21.15.103:0/1322354888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62240 0x56369fb7f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 -- 172.21.15.103:0/1322354888 >> 172.21.15.103:0/1322354888 conn(0x56369fb60020 msgr2=0x56369fb66cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 -- 172.21.15.103:0/1322354888 shutdown_connections 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 -- 172.21.15.103:0/1322354888 wait complete. 2024-09-11T00:23:21.846 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.843+0000 7f86f9018740 1 -- start start 2024-09-11T00:23:21.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 1 Processor -- start 2024-09-11T00:23:21.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62240 0x56369fb83110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb83650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb6eaf0 0x56369fb83b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56369fb80b90 con 0x56369fb6eaf0 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56369fb804c0 con 0x56369fb62b60 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56369fb81260 con 0x56369fb62240 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86f9018740 10 client.0 _ll_register_callbacks cb 0x7f86e000f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb83650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb6eaf0 0x56369fb83b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86e6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb83650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:35774/0 (socket says 172.21.15.103:35774) 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86efe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb6eaf0 0x56369fb83b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33428/0 (socket says 172.21.15.103:33428) 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86e6a00640 1 -- 172.21.15.103:0/913519080 learned_addr learned my addr 172.21.15.103:0/913519080 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.844+0000 7f86ef400640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62240 0x56369fb83110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.849 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86efe00640 1 -- 172.21.15.103:0/913519080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62240 msgr2=0x56369fb83110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86efe00640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62240 0x56369fb83110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86efe00640 1 -- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 msgr2=0x56369fb83650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86efe00640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb83650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86efe00640 1 -- 172.21.15.103:0/913519080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369fb88a50 con 0x56369fb6eaf0 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86e6a00640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb83650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86ef400640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62240 0x56369fb83110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86efe00640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb6eaf0 0x56369fb83b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e600 tx=0x7f86e8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f61861f19d355dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86dd600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86dd600640 1 -- 172.21.15.103:0/913519080 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e801e040 con 0x56369fb6eaf0 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86dd600640 1 -- 172.21.15.103:0/913519080 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f86e800e8d0 con 0x56369fb6eaf0 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.845+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d80cc8a0 con 0x56369fb6eaf0 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86dd600640 1 -- 172.21.15.103:0/913519080 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86e8032050 con 0x56369fb6eaf0 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 10 client.15246 fetch_fsmap learned FSMap version 26 2024-09-11T00:23:21.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56369fbec6d0 con 0x56369fb6eaf0 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86dd600640 1 -- 172.21.15.103:0/913519080 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86d80cc8a0 con 0x56369fb6eaf0 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86dd600640 1 -- 172.21.15.103:0/913519080 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f86e8003af0 con 0x56369fb6eaf0 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 10 client.15246 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 10 client.15246 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 20 client.15246 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fbf10d0 0x56369fbefde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 4 client.15246 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.846+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "status"}) -- 0x56369fad7750 con 0x56369fbf10d0 2024-09-11T00:23:21.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.847+0000 7f86ef400640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fbf10d0 0x56369fbefde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:21.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.847+0000 7f86ef400640 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fbf10d0 0x56369fbefde0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28cfefec33e1cb61 server_cookie=237a22d3c120f353 in_seq=0 out_seq=0 2024-09-11T00:23:21.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.847+0000 7f86dd600640 10 client.15246 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:21.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86dd600640 1 -- 172.21.15.103:0/913519080 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56369fad7750 con 0x56369fbf10d0 2024-09-11T00:23:21.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86dd600640 10 client.15246 handle_command_reply: tid=0 2024-09-11T00:23:21.852 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 client.15246 shutdown 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb6eaf0 msgr2=0x56369fb83b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb6eaf0 0x56369fb83b90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f86e800e600 tx=0x7f86e8005750 comp rx=0 tx=0).stop 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 shutdown_connections 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56369fb6eaf0 0x56369fb83b90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56369fb62b60 0x56369fb83650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56369fbf10d0 0x56369fbefde0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 --2- 172.21.15.103:0/913519080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56369fb62240 0x56369fb83110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 >> 172.21.15.103:0/913519080 conn(0x56369fb60020 msgr2=0x56369fb66cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86dd600640 0 client.15246 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 shutdown_connections 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 1 -- 172.21.15.103:0/913519080 wait complete. 2024-09-11T00:23:21.853 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.848+0000 7f86f9018740 20 client.15246 trim_cache size 0 max 16384 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24656, 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 172.79415536600001, 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 26, 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 86, 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 187.77023761800001 2024-09-11T00:23:21.854 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:23:21.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86d40b52e0 msgr2=0x7f86d40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86d40b52e0 0x7f86d40b77a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x56369fb653c0 tx=0x7f86e0009500 comp rx=0 tx=0).stop 2024-09-11T00:23:21.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f01087d0 msgr2=0x7f86f019b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:21.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f01087d0 0x7f86f019b9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f86d8005ab0 tx=0x7f86d8003c10 comp rx=0 tx=0).stop 2024-09-11T00:23:21.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 shutdown_connections 2024-09-11T00:23:21.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86d40b52e0 0x7f86d40b77a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86f0109150 0x7f86f01a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86f01087d0 0x7f86f019b9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 --2- 172.21.15.103:0/4292342918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86f0102dc0 0x7f86f019b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:21.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 >> 172.21.15.103:0/4292342918 conn(0x7f86f00fe830 msgr2=0x7f86f0105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:21.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 shutdown_connections 2024-09-11T00:23:21.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:21.849+0000 7f86f6400640 1 -- 172.21.15.103:0/4292342918 wait complete. 2024-09-11T00:23:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:22 smithi183 ceph-mon[35934]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:22.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:22 smithi103 ceph-mon[30578]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:22 smithi117 ceph-mon[35525]: pgmap v407: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:22.978 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T00:23:22.991 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T00:23:22.991 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T00:23:22.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:22.992 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-11T00:23:23.029 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-11T00:23:23.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:23.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-11T00:23:23.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:23.129 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-11T00:23:23.436 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-11T00:23:23.451 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T00:23:23.451 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:23.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T00:23:23.826 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.461+0000 7fa1f0600640 1 -- 172.21.15.103:0/1504176809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 msgr2=0x7fa1e810ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.461+0000 7fa1f0600640 1 --2- 172.21.15.103:0/1504176809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 0x7fa1e810ee90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009900 tx=0x7fa1d402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.461+0000 7fa1f0600640 1 -- 172.21.15.103:0/1504176809 shutdown_connections 2024-09-11T00:23:24.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.461+0000 7fa1f0600640 1 --2- 172.21.15.103:0/1504176809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e810f3d0 0x7fa1e81117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.461+0000 7fa1f0600640 1 --2- 172.21.15.103:0/1504176809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 0x7fa1e810ee90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.461+0000 7fa1f0600640 1 --2- 172.21.15.103:0/1504176809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1e80754c0 0x7fa1e80758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.461+0000 7fa1f0600640 1 -- 172.21.15.103:0/1504176809 >> 172.21.15.103:0/1504176809 conn(0x7fa1e80fe830 msgr2=0x7fa1e8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.462+0000 7fa1f0600640 1 -- 172.21.15.103:0/1504176809 shutdown_connections 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.462+0000 7fa1f0600640 1 -- 172.21.15.103:0/1504176809 wait complete. 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.462+0000 7fa1f0600640 1 Processor -- start 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1f0600640 1 -- start start 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1f0600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e80754c0 0x7fa1e819b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1f0600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 0x7fa1e819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1f0600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1e810f3d0 0x7fa1e81a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1f0600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8115010 con 0x7fa1e80754c0 2024-09-11T00:23:24.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1f0600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8114e90 con 0x7fa1e8075de0 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1f0600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8115190 con 0x7fa1e810f3d0 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1edc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e80754c0 0x7fa1e819b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1ee600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1e810f3d0 0x7fa1e81a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1edc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e80754c0 0x7fa1e819b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58468/0 (socket says 172.21.15.103:58468) 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1ee600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1e810f3d0 0x7fa1e81a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:46162/0 (socket says 172.21.15.103:46162) 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1edc00640 1 -- 172.21.15.103:0/3444662930 learned_addr learned my addr 172.21.15.103:0/3444662930 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.463+0000 7fa1ed200640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 0x7fa1e819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1edc00640 1 -- 172.21.15.103:0/3444662930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1e810f3d0 msgr2=0x7fa1e81a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1edc00640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1e810f3d0 0x7fa1e81a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1edc00640 1 -- 172.21.15.103:0/3444662930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 msgr2=0x7fa1e819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1edc00640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 0x7fa1e819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1edc00640 1 -- 172.21.15.103:0/3444662930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e81a15b0 con 0x7fa1e80754c0 2024-09-11T00:23:24.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1edc00640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e80754c0 0x7fa1e819b500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00eb90 tx=0x7fa1dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1e6000640 1 -- 172.21.15.103:0/3444662930 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc004d20 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e81a18a0 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1e6000640 1 -- 172.21.15.103:0/3444662930 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1dc00ee90 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1e6000640 1 -- 172.21.15.103:0/3444662930 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc010670 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.464+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81a7f60 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.465+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e80758a0 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.467+0000 7fa1e6000640 1 -- 172.21.15.103:0/3444662930 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa1dc005020 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.469+0000 7fa1e6000640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1d00b5360 0x7fa1d00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.469+0000 7fa1e6000640 1 -- 172.21.15.103:0/3444662930 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fa1dc0e65a0 con 0x7fa1e80754c0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.469+0000 7fa1ed200640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1d00b5360 0x7fa1d00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.470+0000 7fa1ed200640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1d00b5360 0x7fa1d00b7820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4005ab0 tx=0x7fa1d4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:24.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.473+0000 7fa1e6000640 1 -- 172.21.15.103:0/3444662930 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa1dc0ad070 con 0x7fa1e80754c0 2024-09-11T00:23:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:24 smithi103 ceph-mon[30578]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:24.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.698+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa1e81a82f0 con 0x7fa1e80754c0 2024-09-11T00:23:24.700 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.698+0000 7fa1e6000640 1 -- 172.21.15.103:0/3444662930 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+88 (secure 0 0 0) 0x7fa1dc0ac970 con 0x7fa1e80754c0 2024-09-11T00:23:24.700 INFO:teuthology.orchestra.run.smithi103.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1d00b5360 msgr2=0x7fa1d00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1d00b5360 0x7fa1d00b7820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4005ab0 tx=0x7fa1d4005a20 comp rx=0 tx=0).stop 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e80754c0 msgr2=0x7fa1e819b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e80754c0 0x7fa1e819b500 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc00eb90 tx=0x7fa1dc00c6a0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 shutdown_connections 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1d00b5360 0x7fa1d00b7820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1e810f3d0 0x7fa1e81a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1e8075de0 0x7fa1e819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.703 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 --2- 172.21.15.103:0/3444662930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1e80754c0 0x7fa1e819b500 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:24.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 >> 172.21.15.103:0/3444662930 conn(0x7fa1e80fe830 msgr2=0x7fa1e81115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:24.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 shutdown_connections 2024-09-11T00:23:24.704 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:24.701+0000 7fa1f0600640 1 -- 172.21.15.103:0/3444662930 wait complete. 2024-09-11T00:23:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:24 smithi183 ceph-mon[35934]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:24 smithi117 ceph-mon[35525]: pgmap v408: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T00:23:25.663 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T00:23:25.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:25.664 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:23:25.700 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 41994sec preferred_lft 41994sec 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link noprefixroute 2024-09-11T00:23:25.701 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-09-11T00:23:25.702 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T00:23:25.702 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:23:25.703 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:23:25.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-09-11T00:23:25.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-09-11T00:23:25.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-09-11T00:23:25.703 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T00:23:25.703 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:23:25.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:25 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3444662930' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:23:25.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:23:25.837 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:25 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3444662930' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:23:25.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:25.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:25.928 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T00:23:25.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:25 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3444662930' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-11T00:23:25.983 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-09-11T00:23:25.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:25.987 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-09-11T00:23:26.023 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-09-11T00:23:26.023 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T00:23:26.023 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T00:23:26.023 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T00:23:26.023 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T00:23:26.024 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:23:26.024 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:23:26.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T00:23:26.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T00:23:26.024 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T00:23:26.024 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:23:26.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:23:26.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:26.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:26.222 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-11T00:23:26.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:26.258 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-09-11T00:23:26.336 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:23:26.336 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:23:26.336 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T00:23:26.336 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T00:23:26.336 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:23:26.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:23:26.506 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:26.513 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T00:23:26.514 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:23:26.593 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:23:26.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:26 smithi103 ceph-mon[30578]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:26.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:26.731 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T00:23:26.732 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-09-11T00:23:26.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-09-11T00:23:26.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T00:23:26.732 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-09-11T00:23:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:26 smithi183 ceph-mon[35934]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:26.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:23:26.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:26.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:23:26.899 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:26.935 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T00:23:26.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T00:23:26.935 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:26 smithi117 ceph-mon[35525]: pgmap v409: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-11T00:23:27.009 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T00:23:27.009 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T00:23:27.010 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/e9949dac-966e-4aba-acd3-2eaad7312f6e 2024-09-11T00:23:27.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:27.010 DEBUG:teuthology.orchestra.run.smithi103:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/e9949dac-966e-4aba-acd3-2eaad7312f6e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-11T00:23:27.138 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-11T00:23:27.138 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-11T00:23:27.780 INFO:teuthology.orchestra.run.smithi103.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-11T00:23:27.780 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-11T00:23:27.780 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: resolved to: "172.21.15.103:6789,172.21.15.117:6789,172.21.15.183:6789" 2024-09-11T00:23:27.780 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: trying mount with new device syntax: 0@40f10072-6fd2-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/e9949dac-966e-4aba-acd3-2eaad7312f6e 2024-09-11T00:23:27.780 INFO:teuthology.orchestra.run.smithi103.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.103:6789/172.21.15.117:6789/172.21.15.183:6789" will pass to kernel 2024-09-11T00:23:27.783 INFO:teuthology.orchestra.run.smithi103.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T00:23:27.783 INFO:teuthology.orchestra.run.smithi103.stdout: You just mounted a file system that supports labels which does not 2024-09-11T00:23:27.783 INFO:teuthology.orchestra.run.smithi103.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T00:23:27.783 INFO:teuthology.orchestra.run.smithi103.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T00:23:27.783 INFO:teuthology.orchestra.run.smithi103.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T00:23:27.786 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T00:23:27.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:23:27.786 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:27.845 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:23:27.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:23:27.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:27.937 INFO:teuthology.orchestra.run.smithi103.stdout:client25265 2024-09-11T00:23:27.938 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:23:27.938 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:23:27.975 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:23:28.095 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:23:28.099 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:28.107 INFO:teuthology.orchestra.run.smithi103.stdout:client25265 2024-09-11T00:23:28.108 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T00:23:28.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T00:23:28.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:28.200 INFO:teuthology.orchestra.run.smithi103.stdout:client25265 2024-09-11T00:23:28.201 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/40f10072-6fd2-11ef-bcea-c7b262605968.client25265/status') 2024-09-11T00:23:28.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T00:23:28.300 INFO:teuthology.orchestra.run.smithi103.stdout:instance: client.25265 (3)172.21.15.103:0/1596531492 2024-09-11T00:23:28.300 INFO:teuthology.orchestra.run.smithi103.stdout:blocklisted: false 2024-09-11T00:23:28.300 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records in 2024-09-11T00:23:28.300 INFO:teuthology.orchestra.run.smithi103.stderr:0+1 records out 2024-09-11T00:23:28.301 INFO:teuthology.orchestra.run.smithi103.stderr:72 bytes copied, 5.3962e-05 s, 1.3 MB/s 2024-09-11T00:23:28.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T00:23:28.311 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T00:23:28.319 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T00:23:28.319 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:23:28.319 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.471+0000 7f69c1c00640 1 -- 172.21.15.103:0/1313177751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc075ac0 msgr2=0x7f69bc075f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.471+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1313177751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc075ac0 0x7f69bc075f40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69a4009900 tx=0x7f69a402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.472+0000 7f69c1c00640 1 -- 172.21.15.103:0/1313177751 shutdown_connections 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.472+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1313177751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc10a710 0x7f69bc10cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.472+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1313177751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc075ac0 0x7f69bc075f40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.472+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1313177751 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc076f30 0x7f69bc075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.472+0000 7f69c1c00640 1 -- 172.21.15.103:0/1313177751 >> 172.21.15.103:0/1313177751 conn(0x7f69bc0fe030 msgr2=0x7f69bc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.472+0000 7f69c1c00640 1 -- 172.21.15.103:0/1313177751 shutdown_connections 2024-09-11T00:23:28.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.472+0000 7f69c1c00640 1 -- 172.21.15.103:0/1313177751 wait complete. 2024-09-11T00:23:28.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 Processor -- start 2024-09-11T00:23:28.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 -- start start 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc075ac0 0x7f69bc19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc076f30 0x7f69bc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc10a710 0x7f69bc19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc110600 con 0x7f69bc075ac0 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc110480 con 0x7f69bc076f30 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc110780 con 0x7f69bc10a710 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc075ac0 0x7f69bc19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc075ac0 0x7f69bc19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58490/0 (socket says 172.21.15.103:58490) 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69b3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc076f30 0x7f69bc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69bbe00640 1 -- 172.21.15.103:0/1514880826 learned_addr learned my addr 172.21.15.103:0/1514880826 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.473+0000 7f69c1200640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc10a710 0x7f69bc19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:28.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69bbe00640 1 -- 172.21.15.103:0/1514880826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc10a710 msgr2=0x7f69bc19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69bbe00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc10a710 0x7f69bc19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69bbe00640 1 -- 172.21.15.103:0/1514880826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc076f30 msgr2=0x7f69bc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69bbe00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc076f30 0x7f69bc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69bbe00640 1 -- 172.21.15.103:0/1514880826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc19c920 con 0x7f69bc075ac0 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69bbe00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc075ac0 0x7f69bc19ade0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00d770 tx=0x7f69ac00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69b9600640 1 -- 172.21.15.103:0/1514880826 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac017070 con 0x7f69bc075ac0 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69b9600640 1 -- 172.21.15.103:0/1514880826 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ac009820 con 0x7f69bc075ac0 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc19cc10 con 0x7f69bc075ac0 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69b9600640 1 -- 172.21.15.103:0/1514880826 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac004bc0 con 0x7f69bc075ac0 2024-09-11T00:23:28.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.474+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc1a32e0 con 0x7f69bc075ac0 2024-09-11T00:23:28.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.476+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc109520 con 0x7f69bc075ac0 2024-09-11T00:23:28.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.479+0000 7f69b9600640 1 -- 172.21.15.103:0/1514880826 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f69ac005220 con 0x7f69bc075ac0 2024-09-11T00:23:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.480+0000 7f69b9600640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69a00b52e0 0x7f69a00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.481+0000 7f69b9600640 1 -- 172.21.15.103:0/1514880826 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f69ac0e5670 con 0x7f69bc075ac0 2024-09-11T00:23:28.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.481+0000 7f69b3e00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69a00b52e0 0x7f69a00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:28.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.481+0000 7f69b3e00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69a00b52e0 0x7f69a00b77a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f69a4000c00 tx=0x7f69a40314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:28.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.485+0000 7f69b9600640 1 -- 172.21.15.103:0/1514880826 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f69ac0ac4f0 con 0x7f69bc075ac0 2024-09-11T00:23:28.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:28 smithi103 ceph-mon[30578]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:23:28.708 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:28.707+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f69bc02e5c0 con 0x7f69bc075ac0 2024-09-11T00:23:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:28 smithi183 ceph-mon[35934]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:23:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:28 smithi117 ceph-mon[35525]: pgmap v410: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:23:29.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.487+0000 7f69b9600640 1 -- 172.21.15.103:0/1514880826 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v27) ==== 117+0+0 (secure 0 0 0) 0x7f69ac0abdf0 con 0x7f69bc075ac0 2024-09-11T00:23:29.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.490+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69a00b52e0 msgr2=0x7f69a00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.490+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69a00b52e0 0x7f69a00b77a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f69a4000c00 tx=0x7f69a40314a0 comp rx=0 tx=0).stop 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.490+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc075ac0 msgr2=0x7f69bc19ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.490+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc075ac0 0x7f69bc19ade0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00d770 tx=0x7f69ac00db70 comp rx=0 tx=0).stop 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 shutdown_connections 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69a00b52e0 0x7f69a00b77a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69bc10a710 0x7f69bc19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69bc076f30 0x7f69bc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 --2- 172.21.15.103:0/1514880826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69bc075ac0 0x7f69bc19ade0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 >> 172.21.15.103:0/1514880826 conn(0x7f69bc0fe030 msgr2=0x7f69bc1066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 shutdown_connections 2024-09-11T00:23:29.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:29.491+0000 7f69c1c00640 1 -- 172.21.15.103:0/1514880826 wait complete. 2024-09-11T00:23:29.511 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T00:23:29.521 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T00:23:29.529 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:23:29.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T00:23:29.533 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T00:23:29.544 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T00:23:29.545 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:23:29.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:29 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1514880826' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:23:29.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:29 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1514880826' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:23:29.921 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:29 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1514880826' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-11T00:23:30.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 -- 172.21.15.103:0/1922136865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c8077ff0 msgr2=0x7f47c8113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:30.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 --2- 172.21.15.103:0/1922136865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c8077ff0 0x7f47c8113450 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009900 tx=0x7f47b402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 -- 172.21.15.103:0/1922136865 shutdown_connections 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 --2- 172.21.15.103:0/1922136865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c8113990 0x7f47c8115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 --2- 172.21.15.103:0/1922136865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c8077ff0 0x7f47c8113450 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 --2- 172.21.15.103:0/1922136865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47c80776d0 0x7f47c8077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 -- 172.21.15.103:0/1922136865 >> 172.21.15.103:0/1922136865 conn(0x7f47c8100a50 msgr2=0x7f47c8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 -- 172.21.15.103:0/1922136865 shutdown_connections 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.509+0000 7f47cf200640 1 -- 172.21.15.103:0/1922136865 wait complete. 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.510+0000 7f47cf200640 1 Processor -- start 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.510+0000 7f47cf200640 1 -- start start 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47cf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c80776d0 0x7f47c819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47cf200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47c8077ff0 0x7f47c819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47cf200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c8113990 0x7f47c81a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:30.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47cf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81193e0 con 0x7f47c80776d0 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47cf200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8119260 con 0x7f47c8113990 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47cf200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8119560 con 0x7f47c8077ff0 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c80776d0 0x7f47c819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47c7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47c8077ff0 0x7f47c819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47c7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c80776d0 0x7f47c819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58494/0 (socket says 172.21.15.103:58494) 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47c7e00640 1 -- 172.21.15.103:0/3966823761 learned_addr learned my addr 172.21.15.103:0/3966823761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.511+0000 7f47cd200640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c8113990 0x7f47c81a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c7e00640 1 -- 172.21.15.103:0/3966823761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47c8077ff0 msgr2=0x7f47c819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c7e00640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47c8077ff0 0x7f47c819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c7e00640 1 -- 172.21.15.103:0/3966823761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c8113990 msgr2=0x7f47c81a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c7e00640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c8113990 0x7f47c81a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c7e00640 1 -- 172.21.15.103:0/3966823761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c81a5940 con 0x7f47c80776d0 2024-09-11T00:23:30.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c7e00640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c80776d0 0x7f47c819f890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f47b800e9f0 tx=0x7f47b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:30.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c4c00640 1 -- 172.21.15.103:0/3966823761 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b800cd80 con 0x7f47c80776d0 2024-09-11T00:23:30.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c4c00640 1 -- 172.21.15.103:0/3966823761 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47b80054d0 con 0x7f47c80776d0 2024-09-11T00:23:30.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c81a5c30 con 0x7f47c80776d0 2024-09-11T00:23:30.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.512+0000 7f47c4c00640 1 -- 172.21.15.103:0/3966823761 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8010690 con 0x7f47c80776d0 2024-09-11T00:23:30.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.513+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81ac2f0 con 0x7f47c80776d0 2024-09-11T00:23:30.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.514+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c8077ab0 con 0x7f47c80776d0 2024-09-11T00:23:30.517 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.516+0000 7f47c4c00640 1 -- 172.21.15.103:0/3966823761 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f47b8004e90 con 0x7f47c80776d0 2024-09-11T00:23:30.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.517+0000 7f47c4c00640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f47ac0b52e0 0x7f47ac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:30.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.517+0000 7f47c4c00640 1 -- 172.21.15.103:0/3966823761 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f47b80e63d0 con 0x7f47c80776d0 2024-09-11T00:23:30.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.517+0000 7f47c7400640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f47ac0b52e0 0x7f47ac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:30.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.518+0000 7f47c7400640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f47ac0b52e0 0x7f47ac0b77a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005ab0 tx=0x7f47b4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:30.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.521+0000 7f47c4c00640 1 -- 172.21.15.103:0/3966823761 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f47b80acea0 con 0x7f47c80776d0 2024-09-11T00:23:30.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:30 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1514880826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:23:30.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:30 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:23:30.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:30 smithi103 ceph-mon[30578]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-11T00:23:30.706 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.705+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47c81159e0 con 0x7f47ac0b52e0 2024-09-11T00:23:30.713 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.711+0000 7f47c4c00640 1 -- 172.21.15.103:0/3966823761 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340474 (secure 0 0 0) 0x7f47c81159e0 con 0x7f47ac0b52e0 2024-09-11T00:23:30.717 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:30.719 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f47ac0b52e0 msgr2=0x7f47ac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f47ac0b52e0 0x7f47ac0b77a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005ab0 tx=0x7f47b4005a20 comp rx=0 tx=0).stop 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c80776d0 msgr2=0x7f47c819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c80776d0 0x7f47c819f890 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f47b800e9f0 tx=0x7f47b800edf0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 shutdown_connections 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f47ac0b52e0 0x7f47ac0b77a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47c8113990 0x7f47c81a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f47c8077ff0 0x7f47c819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 --2- 172.21.15.103:0/3966823761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47c80776d0 0x7f47c819f890 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 >> 172.21.15.103:0/3966823761 conn(0x7f47c8100a50 msgr2=0x7f47c8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:30.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 shutdown_connections 2024-09-11T00:23:30.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:30.716+0000 7f47cf200640 1 -- 172.21.15.103:0/3966823761 wait complete. 2024-09-11T00:23:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:30 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1514880826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:23:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:30 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:23:30.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:30 smithi183 ceph-mon[35934]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-11T00:23:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:30 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1514880826' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-11T00:23:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:30 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:23:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:30 smithi117 ceph-mon[35525]: pgmap v411: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-11T00:23:31.617 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-11T00:23:30.407248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657687,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":848,"num_read_kb":520,"num_write":304,"num_write_kb":2277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":747,"ondisk_log_size":747,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375824,"kb_used_data":14096,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566000,"statfs":{"total":1151940427776,"available":1151555584000,"internally_reserved":0,"allocated":14434304,"data_stored":8128441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":42,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003297"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680924+0000","last_change":"2024-09-11T00:20:24.027578+0000","last_active":"2024-09-11T00:22:28.680924+0000","last_peered":"2024-09-11T00:22:28.680924+0000","last_clean":"2024-09-11T00:22:28.680924+0000","last_became_active":"2024-09-11T00:20:24.027174+0000","last_became_peered":"2024-09-11T00:20:24.027174+0000","last_unstale":"2024-09-11T00:22:28.680924+0000","last_undegraded":"2024-09-11T00:22:28.680924+0000","last_fullsized":"2024-09-11T00:22:28.680924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:15.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.595270+0000","last_change":"2024-09-11T00:20:27.064047+0000","last_active":"2024-09-11T00:22:56.595270+0000","last_peered":"2024-09-11T00:22:56.595270+0000","last_clean":"2024-09-11T00:22:56.595270+0000","last_became_active":"2024-09-11T00:20:27.063756+0000","last_became_peered":"2024-09-11T00:20:27.063756+0000","last_unstale":"2024-09-11T00:22:56.595270+0000","last_undegraded":"2024-09-11T00:22:56.595270+0000","last_fullsized":"2024-09-11T00:22:56.595270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.695888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679261+0000","last_change":"2024-09-11T00:20:37.449428+0000","last_active":"2024-09-11T00:22:28.679261+0000","last_peered":"2024-09-11T00:22:28.679261+0000","last_clean":"2024-09-11T00:22:28.679261+0000","last_became_active":"2024-09-11T00:20:37.449015+0000","last_became_peered":"2024-09-11T00:20:37.449015+0000","last_unstale":"2024-09-11T00:22:28.679261+0000","last_undegraded":"2024-09-11T00:22:28.679261+0000","last_fullsized":"2024-09-11T00:22:28.679261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:42.983990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864076+0000","last_change":"2024-09-11T00:20:24.035966+0000","last_active":"2024-09-11T00:22:28.864076+0000","last_peered":"2024-09-11T00:22:28.864076+0000","last_clean":"2024-09-11T00:22:28.864076+0000","last_became_active":"2024-09-11T00:20:24.035663+0000","last_became_peered":"2024-09-11T00:20:24.035663+0000","last_unstale":"2024-09-11T00:22:28.864076+0000","last_undegraded":"2024-09-11T00:22:28.864076+0000","last_fullsized":"2024-09-11T00:22:28.864076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:22.529392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612592+0000","last_change":"2024-09-11T00:20:27.066470+0000","last_active":"2024-09-11T00:22:56.612592+0000","last_peered":"2024-09-11T00:22:56.612592+0000","last_clean":"2024-09-11T00:22:56.612592+0000","last_became_active":"2024-09-11T00:20:27.065424+0000","last_became_peered":"2024-09-11T00:20:27.065424+0000","last_unstale":"2024-09-11T00:22:56.612592+0000","last_undegraded":"2024-09-11T00:22:56.612592+0000","last_fullsized":"2024-09-11T00:22:56.612592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:39.967332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864816+0000","last_change":"2024-09-11T00:20:37.453105+0000","last_active":"2024-09-11T00:22:28.864816+0000","last_peered":"2024-09-11T00:22:28.864816+0000","last_clean":"2024-09-11T00:22:28.864816+0000","last_became_active":"2024-09-11T00:20:37.452319+0000","last_became_peered":"2024-09-11T00:20:37.452319+0000","last_unstale":"2024-09-11T00:22:28.864816+0000","last_undegraded":"2024-09-11T00:22:28.864816+0000","last_fullsized":"2024-09-11T00:22:28.864816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:05.873046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864807+0000","last_change":"2024-09-11T00:20:24.022277+0000","last_active":"2024-09-11T00:22:28.864807+0000","last_peered":"2024-09-11T00:22:28.864807+0000","last_clean":"2024-09-11T00:22:28.864807+0000","last_became_active":"2024-09-11T00:20:24.021893+0000","last_became_peered":"2024-09-11T00:20:24.021893+0000","last_unstale":"2024-09-11T00:22:28.864807+0000","last_undegraded":"2024-09-11T00:22:28.864807+0000","last_fullsized":"2024-09-11T00:22:28.864807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.994032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613024+0000","last_change":"2024-09-11T00:20:27.063368+0000","last_active":"2024-09-11T00:22:56.613024+0000","last_peered":"2024-09-11T00:22:56.613024+0000","last_clean":"2024-09-11T00:22:56.613024+0000","last_became_active":"2024-09-11T00:20:27.063051+0000","last_became_peered":"2024-09-11T00:20:27.063051+0000","last_unstale":"2024-09-11T00:22:56.613024+0000","last_undegraded":"2024-09-11T00:22:56.613024+0000","last_fullsized":"2024-09-11T00:22:56.613024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:09.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864842+0000","last_change":"2024-09-11T00:21:28.229073+0000","last_active":"2024-09-11T00:22:28.864842+0000","last_peered":"2024-09-11T00:22:28.864842+0000","last_clean":"2024-09-11T00:22:28.864842+0000","last_became_active":"2024-09-11T00:21:28.228775+0000","last_became_peered":"2024-09-11T00:21:28.228775+0000","last_unstale":"2024-09-11T00:22:28.864842+0000","last_undegraded":"2024-09-11T00:22:28.864842+0000","last_fullsized":"2024-09-11T00:22:28.864842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.886727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864759+0000","last_change":"2024-09-11T00:20:24.022562+0000","last_active":"2024-09-11T00:22:28.864759+0000","last_peered":"2024-09-11T00:22:28.864759+0000","last_clean":"2024-09-11T00:22:28.864759+0000","last_became_active":"2024-09-11T00:20:24.022207+0000","last_became_peered":"2024-09-11T00:20:24.022207+0000","last_unstale":"2024-09-11T00:22:28.864759+0000","last_undegraded":"2024-09-11T00:22:28.864759+0000","last_fullsized":"2024-09-11T00:22:28.864759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:01.071034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608323+0000","last_change":"2024-09-11T00:20:27.062572+0000","last_active":"2024-09-11T00:22:56.608323+0000","last_peered":"2024-09-11T00:22:56.608323+0000","last_clean":"2024-09-11T00:22:56.608323+0000","last_became_active":"2024-09-11T00:20:27.062288+0000","last_became_peered":"2024-09-11T00:20:27.062288+0000","last_unstale":"2024-09-11T00:22:56.608323+0000","last_undegraded":"2024-09-11T00:22:56.608323+0000","last_fullsized":"2024-09-11T00:22:56.608323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:37.391822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864680+0000","last_change":"2024-09-11T00:20:37.444028+0000","last_active":"2024-09-11T00:22:28.864680+0000","last_peered":"2024-09-11T00:22:28.864680+0000","last_clean":"2024-09-11T00:22:28.864680+0000","last_became_active":"2024-09-11T00:20:37.443344+0000","last_became_peered":"2024-09-11T00:20:37.443344+0000","last_unstale":"2024-09-11T00:22:28.864680+0000","last_undegraded":"2024-09-11T00:22:28.864680+0000","last_fullsized":"2024-09-11T00:22:28.864680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.347054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611869+0000","last_change":"2024-09-11T00:20:27.060317+0000","last_active":"2024-09-11T00:22:56.611869+0000","last_peered":"2024-09-11T00:22:56.611869+0000","last_clean":"2024-09-11T00:22:56.611869+0000","last_became_active":"2024-09-11T00:20:27.059983+0000","last_became_peered":"2024-09-11T00:20:27.059983+0000","last_unstale":"2024-09-11T00:22:56.611869+0000","last_undegraded":"2024-09-11T00:22:56.611869+0000","last_fullsized":"2024-09-11T00:22:56.611869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:28.405553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864258+0000","last_change":"2024-09-11T00:20:24.025555+0000","last_active":"2024-09-11T00:22:28.864258+0000","last_peered":"2024-09-11T00:22:28.864258+0000","last_clean":"2024-09-11T00:22:28.864258+0000","last_became_active":"2024-09-11T00:20:24.025171+0000","last_became_peered":"2024-09-11T00:20:24.025171+0000","last_unstale":"2024-09-11T00:22:28.864258+0000","last_undegraded":"2024-09-11T00:22:28.864258+0000","last_fullsized":"2024-09-11T00:22:28.864258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:22.708985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864496+0000","last_change":"2024-09-11T00:20:37.442057+0000","last_active":"2024-09-11T00:22:28.864496+0000","last_peered":"2024-09-11T00:22:28.864496+0000","last_clean":"2024-09-11T00:22:28.864496+0000","last_became_active":"2024-09-11T00:20:37.441555+0000","last_became_peered":"2024-09-11T00:20:37.441555+0000","last_unstale":"2024-09-11T00:22:28.864496+0000","last_undegraded":"2024-09-11T00:22:28.864496+0000","last_fullsized":"2024-09-11T00:22:28.864496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:08.155763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611677+0000","last_change":"2024-09-11T00:20:27.071533+0000","last_active":"2024-09-11T00:22:56.611677+0000","last_peered":"2024-09-11T00:22:56.611677+0000","last_clean":"2024-09-11T00:22:56.611677+0000","last_became_active":"2024-09-11T00:20:27.068771+0000","last_became_peered":"2024-09-11T00:20:27.068771+0000","last_unstale":"2024-09-11T00:22:56.611677+0000","last_undegraded":"2024-09-11T00:22:56.611677+0000","last_fullsized":"2024-09-11T00:22:56.611677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:05.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679914+0000","last_change":"2024-09-11T00:20:24.023282+0000","last_active":"2024-09-11T00:22:28.679914+0000","last_peered":"2024-09-11T00:22:28.679914+0000","last_clean":"2024-09-11T00:22:28.679914+0000","last_became_active":"2024-09-11T00:20:24.022914+0000","last_became_peered":"2024-09-11T00:20:24.022914+0000","last_unstale":"2024-09-11T00:22:28.679914+0000","last_undegraded":"2024-09-11T00:22:28.679914+0000","last_fullsized":"2024-09-11T00:22:28.679914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:45.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864811+0000","last_change":"2024-09-11T00:20:37.457338+0000","last_active":"2024-09-11T00:22:28.864811+0000","last_peered":"2024-09-11T00:22:28.864811+0000","last_clean":"2024-09-11T00:22:28.864811+0000","last_became_active":"2024-09-11T00:20:37.456993+0000","last_became_peered":"2024-09-11T00:20:37.456993+0000","last_unstale":"2024-09-11T00:22:28.864811+0000","last_undegraded":"2024-09-11T00:22:28.864811+0000","last_fullsized":"2024-09-11T00:22:28.864811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:57.284412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606431+0000","last_change":"2024-09-11T00:20:27.062752+0000","last_active":"2024-09-11T00:22:56.606431+0000","last_peered":"2024-09-11T00:22:56.606431+0000","last_clean":"2024-09-11T00:22:56.606431+0000","last_became_active":"2024-09-11T00:20:27.061569+0000","last_became_peered":"2024-09-11T00:20:27.061569+0000","last_unstale":"2024-09-11T00:22:56.606431+0000","last_undegraded":"2024-09-11T00:22:56.606431+0000","last_fullsized":"2024-09-11T00:22:56.606431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:30.904423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864019+0000","last_change":"2024-09-11T00:20:24.023197+0000","last_active":"2024-09-11T00:22:28.864019+0000","last_peered":"2024-09-11T00:22:28.864019+0000","last_clean":"2024-09-11T00:22:28.864019+0000","last_became_active":"2024-09-11T00:20:24.022867+0000","last_became_peered":"2024-09-11T00:20:24.022867+0000","last_unstale":"2024-09-11T00:22:28.864019+0000","last_undegraded":"2024-09-11T00:22:28.864019+0000","last_fullsized":"2024-09-11T00:22:28.864019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:37.949437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680695+0000","last_change":"2024-09-11T00:20:37.451731+0000","last_active":"2024-09-11T00:22:28.680695+0000","last_peered":"2024-09-11T00:22:28.680695+0000","last_clean":"2024-09-11T00:22:28.680695+0000","last_became_active":"2024-09-11T00:20:37.447710+0000","last_became_peered":"2024-09-11T00:20:37.447710+0000","last_unstale":"2024-09-11T00:22:28.680695+0000","last_undegraded":"2024-09-11T00:22:28.680695+0000","last_fullsized":"2024-09-11T00:22:28.680695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:11:38.471216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.565404+0000","last_change":"2024-09-11T00:20:27.066021+0000","last_active":"2024-09-11T00:22:56.565404+0000","last_peered":"2024-09-11T00:22:56.565404+0000","last_clean":"2024-09-11T00:22:56.565404+0000","last_became_active":"2024-09-11T00:20:27.065652+0000","last_became_peered":"2024-09-11T00:20:27.065652+0000","last_unstale":"2024-09-11T00:22:56.565404+0000","last_undegraded":"2024-09-11T00:22:56.565404+0000","last_fullsized":"2024-09-11T00:22:56.565404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:01.933031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864139+0000","last_change":"2024-09-11T00:20:24.035619+0000","last_active":"2024-09-11T00:22:28.864139+0000","last_peered":"2024-09-11T00:22:28.864139+0000","last_clean":"2024-09-11T00:22:28.864139+0000","last_became_active":"2024-09-11T00:20:24.035358+0000","last_became_peered":"2024-09-11T00:20:24.035358+0000","last_unstale":"2024-09-11T00:22:28.864139+0000","last_undegraded":"2024-09-11T00:22:28.864139+0000","last_fullsized":"2024-09-11T00:22:28.864139+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:56.890058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865646+0000","last_change":"2024-09-11T00:21:28.226415+0000","last_active":"2024-09-11T00:22:28.865646+0000","last_peered":"2024-09-11T00:22:28.865646+0000","last_clean":"2024-09-11T00:22:28.865646+0000","last_became_active":"2024-09-11T00:21:28.225894+0000","last_became_peered":"2024-09-11T00:21:28.225894+0000","last_unstale":"2024-09-11T00:22:28.865646+0000","last_undegraded":"2024-09-11T00:22:28.865646+0000","last_fullsized":"2024-09-11T00:22:28.865646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:59.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606794+0000","last_change":"2024-09-11T00:22:28.866891+0000","last_active":"2024-09-11T00:22:56.606794+0000","last_peered":"2024-09-11T00:22:56.606794+0000","last_clean":"2024-09-11T00:22:56.606794+0000","last_became_active":"2024-09-11T00:22:28.866541+0000","last_became_peered":"2024-09-11T00:22:28.866541+0000","last_unstale":"2024-09-11T00:22:56.606794+0000","last_undegraded":"2024-09-11T00:22:56.606794+0000","last_fullsized":"2024-09-11T00:22:56.606794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:35.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865567+0000","last_change":"2024-09-11T00:20:24.535701+0000","last_active":"2024-09-11T00:22:28.865567+0000","last_peered":"2024-09-11T00:22:28.865567+0000","last_clean":"2024-09-11T00:22:28.865567+0000","last_became_active":"2024-09-11T00:20:24.535389+0000","last_became_peered":"2024-09-11T00:20:24.535389+0000","last_unstale":"2024-09-11T00:22:28.865567+0000","last_undegraded":"2024-09-11T00:22:28.865567+0000","last_fullsized":"2024-09-11T00:22:28.865567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:29.044747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679639+0000","last_change":"2024-09-11T00:20:37.435584+0000","last_active":"2024-09-11T00:22:28.679639+0000","last_peered":"2024-09-11T00:22:28.679639+0000","last_clean":"2024-09-11T00:22:28.679639+0000","last_became_active":"2024-09-11T00:20:37.435237+0000","last_became_peered":"2024-09-11T00:20:37.435237+0000","last_unstale":"2024-09-11T00:22:28.679639+0000","last_undegraded":"2024-09-11T00:22:28.679639+0000","last_fullsized":"2024-09-11T00:22:28.679639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:34.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610308+0000","last_change":"2024-09-11T00:20:27.068718+0000","last_active":"2024-09-11T00:22:56.610308+0000","last_peered":"2024-09-11T00:22:56.610308+0000","last_clean":"2024-09-11T00:22:56.610308+0000","last_became_active":"2024-09-11T00:20:27.058921+0000","last_became_peered":"2024-09-11T00:20:27.058921+0000","last_unstale":"2024-09-11T00:22:56.610308+0000","last_undegraded":"2024-09-11T00:22:56.610308+0000","last_fullsized":"2024-09-11T00:22:56.610308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:16.056742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":58,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:37.230160+0000","last_change":"2024-09-11T00:20:24.023580+0000","last_active":"2024-09-11T00:22:37.230160+0000","last_peered":"2024-09-11T00:22:37.230160+0000","last_clean":"2024-09-11T00:22:37.230160+0000","last_became_active":"2024-09-11T00:20:24.023029+0000","last_became_peered":"2024-09-11T00:20:24.023029+0000","last_unstale":"2024-09-11T00:22:37.230160+0000","last_undegraded":"2024-09-11T00:22:37.230160+0000","last_fullsized":"2024-09-11T00:22:37.230160+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:05.574228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863468+0000","last_change":"2024-09-11T00:20:37.449229+0000","last_active":"2024-09-11T00:22:28.863468+0000","last_peered":"2024-09-11T00:22:28.863468+0000","last_clean":"2024-09-11T00:22:28.863468+0000","last_became_active":"2024-09-11T00:20:37.447464+0000","last_became_peered":"2024-09-11T00:20:37.447464+0000","last_unstale":"2024-09-11T00:22:28.863468+0000","last_undegraded":"2024-09-11T00:22:28.863468+0000","last_fullsized":"2024-09-11T00:22:28.863468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:47.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597838+0000","last_change":"2024-09-11T00:20:27.063518+0000","last_active":"2024-09-11T00:22:56.597838+0000","last_peered":"2024-09-11T00:22:56.597838+0000","last_clean":"2024-09-11T00:22:56.597838+0000","last_became_active":"2024-09-11T00:20:27.063122+0000","last_became_peered":"2024-09-11T00:20:27.063122+0000","last_unstale":"2024-09-11T00:22:56.597838+0000","last_undegraded":"2024-09-11T00:22:56.597838+0000","last_fullsized":"2024-09-11T00:22:56.597838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:52.269862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864785+0000","last_change":"2024-09-11T00:20:24.035835+0000","last_active":"2024-09-11T00:22:28.864785+0000","last_peered":"2024-09-11T00:22:28.864785+0000","last_clean":"2024-09-11T00:22:28.864785+0000","last_became_active":"2024-09-11T00:20:24.035471+0000","last_became_peered":"2024-09-11T00:20:24.035471+0000","last_unstale":"2024-09-11T00:22:28.864785+0000","last_undegraded":"2024-09-11T00:22:28.864785+0000","last_fullsized":"2024-09-11T00:22:28.864785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:37.114863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863908+0000","last_change":"2024-09-11T00:20:37.439609+0000","last_active":"2024-09-11T00:22:28.863908+0000","last_peered":"2024-09-11T00:22:28.863908+0000","last_clean":"2024-09-11T00:22:28.863908+0000","last_became_active":"2024-09-11T00:20:37.439139+0000","last_became_peered":"2024-09-11T00:20:37.439139+0000","last_unstale":"2024-09-11T00:22:28.863908+0000","last_undegraded":"2024-09-11T00:22:28.863908+0000","last_fullsized":"2024-09-11T00:22:28.863908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:19.152096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610140+0000","last_change":"2024-09-11T00:20:27.062879+0000","last_active":"2024-09-11T00:22:56.610140+0000","last_peered":"2024-09-11T00:22:56.610140+0000","last_clean":"2024-09-11T00:22:56.610140+0000","last_became_active":"2024-09-11T00:20:27.062200+0000","last_became_peered":"2024-09-11T00:20:27.062200+0000","last_unstale":"2024-09-11T00:22:56.610140+0000","last_undegraded":"2024-09-11T00:22:56.610140+0000","last_fullsized":"2024-09-11T00:22:56.610140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:57.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680751+0000","last_change":"2024-09-11T00:21:28.230802+0000","last_active":"2024-09-11T00:22:28.680751+0000","last_peered":"2024-09-11T00:22:28.680751+0000","last_clean":"2024-09-11T00:22:28.680751+0000","last_became_active":"2024-09-11T00:21:28.230378+0000","last_became_peered":"2024-09-11T00:21:28.230378+0000","last_unstale":"2024-09-11T00:22:28.680751+0000","last_undegraded":"2024-09-11T00:22:28.680751+0000","last_fullsized":"2024-09-11T00:22:28.680751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:35.676964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864428+0000","last_change":"2024-09-11T00:20:37.444912+0000","last_active":"2024-09-11T00:22:28.864428+0000","last_peered":"2024-09-11T00:22:28.864428+0000","last_clean":"2024-09-11T00:22:28.864428+0000","last_became_active":"2024-09-11T00:20:37.444472+0000","last_became_peered":"2024-09-11T00:20:37.444472+0000","last_unstale":"2024-09-11T00:22:28.864428+0000","last_undegraded":"2024-09-11T00:22:28.864428+0000","last_fullsized":"2024-09-11T00:22:28.864428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:23.263981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606705+0000","last_change":"2024-09-11T00:20:27.060477+0000","last_active":"2024-09-11T00:22:56.606705+0000","last_peered":"2024-09-11T00:22:56.606705+0000","last_clean":"2024-09-11T00:22:56.606705+0000","last_became_active":"2024-09-11T00:20:27.060128+0000","last_became_peered":"2024-09-11T00:20:27.060128+0000","last_unstale":"2024-09-11T00:22:56.606705+0000","last_undegraded":"2024-09-11T00:22:56.606705+0000","last_fullsized":"2024-09-11T00:22:56.606705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:55.967799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866156+0000","last_change":"2024-09-11T00:20:24.028229+0000","last_active":"2024-09-11T00:22:28.866156+0000","last_peered":"2024-09-11T00:22:28.866156+0000","last_clean":"2024-09-11T00:22:28.866156+0000","last_became_active":"2024-09-11T00:20:24.027832+0000","last_became_peered":"2024-09-11T00:20:24.027832+0000","last_unstale":"2024-09-11T00:22:28.866156+0000","last_undegraded":"2024-09-11T00:22:28.866156+0000","last_fullsized":"2024-09-11T00:22:28.866156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:19.960403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682873+0000","last_change":"2024-09-11T00:20:37.442414+0000","last_active":"2024-09-11T00:22:28.682873+0000","last_peered":"2024-09-11T00:22:28.682873+0000","last_clean":"2024-09-11T00:22:28.682873+0000","last_became_active":"2024-09-11T00:20:37.442021+0000","last_became_peered":"2024-09-11T00:20:37.442021+0000","last_unstale":"2024-09-11T00:22:28.682873+0000","last_undegraded":"2024-09-11T00:22:28.682873+0000","last_fullsized":"2024-09-11T00:22:28.682873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:21.296638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.592369+0000","last_change":"2024-09-11T00:21:28.233303+0000","last_active":"2024-09-11T00:22:56.592369+0000","last_peered":"2024-09-11T00:22:56.592369+0000","last_clean":"2024-09-11T00:22:56.592369+0000","last_became_active":"2024-09-11T00:21:28.232039+0000","last_became_peered":"2024-09-11T00:21:28.232039+0000","last_unstale":"2024-09-11T00:22:56.592369+0000","last_undegraded":"2024-09-11T00:22:56.592369+0000","last_fullsized":"2024-09-11T00:22:56.592369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:36.828102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":170,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:24.395304+0000","last_change":"2024-09-11T00:20:24.037218+0000","last_active":"2024-09-11T00:23:24.395304+0000","last_peered":"2024-09-11T00:23:24.395304+0000","last_clean":"2024-09-11T00:23:24.395304+0000","last_became_active":"2024-09-11T00:20:24.036005+0000","last_became_peered":"2024-09-11T00:20:24.036005+0000","last_unstale":"2024-09-11T00:23:24.395304+0000","last_undegraded":"2024-09-11T00:23:24.395304+0000","last_fullsized":"2024-09-11T00:23:24.395304+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:17.758812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683067+0000","last_change":"2024-09-11T00:20:37.445584+0000","last_active":"2024-09-11T00:22:28.683067+0000","last_peered":"2024-09-11T00:22:28.683067+0000","last_clean":"2024-09-11T00:22:28.683067+0000","last_became_active":"2024-09-11T00:20:37.445196+0000","last_became_peered":"2024-09-11T00:20:37.445196+0000","last_unstale":"2024-09-11T00:22:28.683067+0000","last_undegraded":"2024-09-11T00:22:28.683067+0000","last_fullsized":"2024-09-11T00:22:28.683067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:05.966282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.598168+0000","last_change":"2024-09-11T00:20:27.060428+0000","last_active":"2024-09-11T00:22:56.598168+0000","last_peered":"2024-09-11T00:22:56.598168+0000","last_clean":"2024-09-11T00:22:56.598168+0000","last_became_active":"2024-09-11T00:20:27.060057+0000","last_became_peered":"2024-09-11T00:20:27.060057+0000","last_unstale":"2024-09-11T00:22:56.598168+0000","last_undegraded":"2024-09-11T00:22:56.598168+0000","last_fullsized":"2024-09-11T00:22:56.598168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:37.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865091+0000","last_change":"2024-09-11T00:20:24.025496+0000","last_active":"2024-09-11T00:22:28.865091+0000","last_peered":"2024-09-11T00:22:28.865091+0000","last_clean":"2024-09-11T00:22:28.865091+0000","last_became_active":"2024-09-11T00:20:24.025158+0000","last_became_peered":"2024-09-11T00:20:24.025158+0000","last_unstale":"2024-09-11T00:22:28.865091+0000","last_undegraded":"2024-09-11T00:22:28.865091+0000","last_fullsized":"2024-09-11T00:22:28.865091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:40.439708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734460+0000","last_change":"2024-09-11T00:20:37.453118+0000","last_active":"2024-09-11T00:22:28.734460+0000","last_peered":"2024-09-11T00:22:28.734460+0000","last_clean":"2024-09-11T00:22:28.734460+0000","last_became_active":"2024-09-11T00:20:37.452669+0000","last_became_peered":"2024-09-11T00:20:37.452669+0000","last_unstale":"2024-09-11T00:22:28.734460+0000","last_undegraded":"2024-09-11T00:22:28.734460+0000","last_fullsized":"2024-09-11T00:22:28.734460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:51:58.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606977+0000","last_change":"2024-09-11T00:20:27.069708+0000","last_active":"2024-09-11T00:22:56.606977+0000","last_peered":"2024-09-11T00:22:56.606977+0000","last_clean":"2024-09-11T00:22:56.606977+0000","last_became_active":"2024-09-11T00:20:27.067500+0000","last_became_peered":"2024-09-11T00:20:27.067500+0000","last_unstale":"2024-09-11T00:22:56.606977+0000","last_undegraded":"2024-09-11T00:22:56.606977+0000","last_fullsized":"2024-09-11T00:22:56.606977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:57.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864981+0000","last_change":"2024-09-11T00:20:24.023504+0000","last_active":"2024-09-11T00:22:28.864981+0000","last_peered":"2024-09-11T00:22:28.864981+0000","last_clean":"2024-09-11T00:22:28.864981+0000","last_became_active":"2024-09-11T00:20:24.022706+0000","last_became_peered":"2024-09-11T00:20:24.022706+0000","last_unstale":"2024-09-11T00:22:28.864981+0000","last_undegraded":"2024-09-11T00:22:28.864981+0000","last_fullsized":"2024-09-11T00:22:28.864981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:21.216325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865351+0000","last_change":"2024-09-11T00:19:05.531191+0000","last_active":"2024-09-11T00:22:28.865351+0000","last_peered":"2024-09-11T00:22:28.865351+0000","last_clean":"2024-09-11T00:22:28.865351+0000","last_became_active":"2024-09-11T00:19:05.525664+0000","last_became_peered":"2024-09-11T00:19:05.525664+0000","last_unstale":"2024-09-11T00:22:28.865351+0000","last_undegraded":"2024-09-11T00:22:28.865351+0000","last_fullsized":"2024-09-11T00:22:28.865351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_clean_scrub_stamp":"2024-09-11T00:15:23.365345+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:06.253537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866037+0000","last_change":"2024-09-11T00:20:37.446403+0000","last_active":"2024-09-11T00:22:28.866037+0000","last_peered":"2024-09-11T00:22:28.866037+0000","last_clean":"2024-09-11T00:22:28.866037+0000","last_became_active":"2024-09-11T00:20:37.446029+0000","last_became_peered":"2024-09-11T00:20:37.446029+0000","last_unstale":"2024-09-11T00:22:28.866037+0000","last_undegraded":"2024-09-11T00:22:28.866037+0000","last_fullsized":"2024-09-11T00:22:28.866037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611496+0000","last_change":"2024-09-11T00:20:27.063086+0000","last_active":"2024-09-11T00:22:56.611496+0000","last_peered":"2024-09-11T00:22:56.611496+0000","last_clean":"2024-09-11T00:22:56.611496+0000","last_became_active":"2024-09-11T00:20:27.062790+0000","last_became_peered":"2024-09-11T00:20:27.062790+0000","last_unstale":"2024-09-11T00:22:56.611496+0000","last_undegraded":"2024-09-11T00:22:56.611496+0000","last_fullsized":"2024-09-11T00:22:56.611496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:36.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679645+0000","last_change":"2024-09-11T00:20:24.535837+0000","last_active":"2024-09-11T00:22:28.679645+0000","last_peered":"2024-09-11T00:22:28.679645+0000","last_clean":"2024-09-11T00:22:28.679645+0000","last_became_active":"2024-09-11T00:20:24.535308+0000","last_became_peered":"2024-09-11T00:20:24.535308+0000","last_unstale":"2024-09-11T00:22:28.679645+0000","last_undegraded":"2024-09-11T00:22:28.679645+0000","last_fullsized":"2024-09-11T00:22:28.679645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.807570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864240+0000","last_change":"2024-09-11T00:20:37.443844+0000","last_active":"2024-09-11T00:22:28.864240+0000","last_peered":"2024-09-11T00:22:28.864240+0000","last_clean":"2024-09-11T00:22:28.864240+0000","last_became_active":"2024-09-11T00:20:37.443374+0000","last_became_peered":"2024-09-11T00:20:37.443374+0000","last_unstale":"2024-09-11T00:22:28.864240+0000","last_undegraded":"2024-09-11T00:22:28.864240+0000","last_fullsized":"2024-09-11T00:22:28.864240+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:47.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610053+0000","last_change":"2024-09-11T00:20:27.072874+0000","last_active":"2024-09-11T00:22:56.610053+0000","last_peered":"2024-09-11T00:22:56.610053+0000","last_clean":"2024-09-11T00:22:56.610053+0000","last_became_active":"2024-09-11T00:20:27.072454+0000","last_became_peered":"2024-09-11T00:20:27.072454+0000","last_unstale":"2024-09-11T00:22:56.610053+0000","last_undegraded":"2024-09-11T00:22:56.610053+0000","last_fullsized":"2024-09-11T00:22:56.610053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:25.731037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734678+0000","last_change":"2024-09-11T00:20:24.018929+0000","last_active":"2024-09-11T00:22:28.734678+0000","last_peered":"2024-09-11T00:22:28.734678+0000","last_clean":"2024-09-11T00:22:28.734678+0000","last_became_active":"2024-09-11T00:20:24.018575+0000","last_became_peered":"2024-09-11T00:20:24.018575+0000","last_unstale":"2024-09-11T00:22:28.734678+0000","last_undegraded":"2024-09-11T00:22:28.734678+0000","last_fullsized":"2024-09-11T00:22:28.734678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.695561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679083+0000","last_change":"2024-09-11T00:20:37.453468+0000","last_active":"2024-09-11T00:22:28.679083+0000","last_peered":"2024-09-11T00:22:28.679083+0000","last_clean":"2024-09-11T00:22:28.679083+0000","last_became_active":"2024-09-11T00:20:37.453007+0000","last_became_peered":"2024-09-11T00:20:37.453007+0000","last_unstale":"2024-09-11T00:22:28.679083+0000","last_undegraded":"2024-09-11T00:22:28.679083+0000","last_fullsized":"2024-09-11T00:22:28.679083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:22.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604628+0000","last_change":"2024-09-11T00:20:27.064618+0000","last_active":"2024-09-11T00:22:56.604628+0000","last_peered":"2024-09-11T00:22:56.604628+0000","last_clean":"2024-09-11T00:22:56.604628+0000","last_became_active":"2024-09-11T00:20:27.064310+0000","last_became_peered":"2024-09-11T00:20:27.064310+0000","last_unstale":"2024-09-11T00:22:56.604628+0000","last_undegraded":"2024-09-11T00:22:56.604628+0000","last_fullsized":"2024-09-11T00:22:56.604628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:41.320561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683037+0000","last_change":"2024-09-11T00:20:24.023522+0000","last_active":"2024-09-11T00:22:28.683037+0000","last_peered":"2024-09-11T00:22:28.683037+0000","last_clean":"2024-09-11T00:22:28.683037+0000","last_became_active":"2024-09-11T00:20:24.022694+0000","last_became_peered":"2024-09-11T00:20:24.022694+0000","last_unstale":"2024-09-11T00:22:28.683037+0000","last_undegraded":"2024-09-11T00:22:28.683037+0000","last_fullsized":"2024-09-11T00:22:28.683037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:12.374179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733919+0000","last_change":"2024-09-11T00:20:37.441239+0000","last_active":"2024-09-11T00:22:28.733919+0000","last_peered":"2024-09-11T00:22:28.733919+0000","last_clean":"2024-09-11T00:22:28.733919+0000","last_became_active":"2024-09-11T00:20:37.440825+0000","last_became_peered":"2024-09-11T00:20:37.440825+0000","last_unstale":"2024-09-11T00:22:28.733919+0000","last_undegraded":"2024-09-11T00:22:28.733919+0000","last_fullsized":"2024-09-11T00:22:28.733919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:04.972569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594629+0000","last_change":"2024-09-11T00:20:27.059780+0000","last_active":"2024-09-11T00:22:56.594629+0000","last_peered":"2024-09-11T00:22:56.594629+0000","last_clean":"2024-09-11T00:22:56.594629+0000","last_became_active":"2024-09-11T00:20:27.059411+0000","last_became_peered":"2024-09-11T00:20:27.059411+0000","last_unstale":"2024-09-11T00:22:56.594629+0000","last_undegraded":"2024-09-11T00:22:56.594629+0000","last_fullsized":"2024-09-11T00:22:56.594629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:42.058883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864514+0000","last_change":"2024-09-11T00:20:24.024115+0000","last_active":"2024-09-11T00:22:28.864514+0000","last_peered":"2024-09-11T00:22:28.864514+0000","last_clean":"2024-09-11T00:22:28.864514+0000","last_became_active":"2024-09-11T00:20:24.023729+0000","last_became_peered":"2024-09-11T00:20:24.023729+0000","last_unstale":"2024-09-11T00:22:28.864514+0000","last_undegraded":"2024-09-11T00:22:28.864514+0000","last_fullsized":"2024-09-11T00:22:28.864514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:15.874168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733904+0000","last_change":"2024-09-11T00:20:37.443591+0000","last_active":"2024-09-11T00:22:28.733904+0000","last_peered":"2024-09-11T00:22:28.733904+0000","last_clean":"2024-09-11T00:22:28.733904+0000","last_became_active":"2024-09-11T00:20:37.443115+0000","last_became_peered":"2024-09-11T00:20:37.443115+0000","last_unstale":"2024-09-11T00:22:28.733904+0000","last_undegraded":"2024-09-11T00:22:28.733904+0000","last_fullsized":"2024-09-11T00:22:28.733904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:34.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607526+0000","last_change":"2024-09-11T00:20:27.061676+0000","last_active":"2024-09-11T00:22:56.607526+0000","last_peered":"2024-09-11T00:22:56.607526+0000","last_clean":"2024-09-11T00:22:56.607526+0000","last_became_active":"2024-09-11T00:20:27.061372+0000","last_became_peered":"2024-09-11T00:20:27.061372+0000","last_unstale":"2024-09-11T00:22:56.607526+0000","last_undegraded":"2024-09-11T00:22:56.607526+0000","last_fullsized":"2024-09-11T00:22:56.607526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:24.296673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863776+0000","last_change":"2024-09-11T00:21:28.244227+0000","last_active":"2024-09-11T00:22:28.863776+0000","last_peered":"2024-09-11T00:22:28.863776+0000","last_clean":"2024-09-11T00:22:28.863776+0000","last_became_active":"2024-09-11T00:21:28.231547+0000","last_became_peered":"2024-09-11T00:21:28.231547+0000","last_unstale":"2024-09-11T00:22:28.863776+0000","last_undegraded":"2024-09-11T00:22:28.863776+0000","last_fullsized":"2024-09-11T00:22:28.863776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:35.119308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682992+0000","last_change":"2024-09-11T00:20:37.454603+0000","last_active":"2024-09-11T00:22:28.682992+0000","last_peered":"2024-09-11T00:22:28.682992+0000","last_clean":"2024-09-11T00:22:28.682992+0000","last_became_active":"2024-09-11T00:20:37.454123+0000","last_became_peered":"2024-09-11T00:20:37.454123+0000","last_unstale":"2024-09-11T00:22:28.682992+0000","last_undegraded":"2024-09-11T00:22:28.682992+0000","last_fullsized":"2024-09-11T00:22:28.682992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:09.111768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594320+0000","last_change":"2024-09-11T00:20:27.058806+0000","last_active":"2024-09-11T00:22:56.594320+0000","last_peered":"2024-09-11T00:22:56.594320+0000","last_clean":"2024-09-11T00:22:56.594320+0000","last_became_active":"2024-09-11T00:20:27.058458+0000","last_became_peered":"2024-09-11T00:20:27.058458+0000","last_unstale":"2024-09-11T00:22:56.594320+0000","last_undegraded":"2024-09-11T00:22:56.594320+0000","last_fullsized":"2024-09-11T00:22:56.594320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:48.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:28.401683+0000","last_change":"2024-09-11T00:20:24.022328+0000","last_active":"2024-09-11T00:23:28.401683+0000","last_peered":"2024-09-11T00:23:28.401683+0000","last_clean":"2024-09-11T00:23:28.401683+0000","last_became_active":"2024-09-11T00:20:24.021908+0000","last_became_peered":"2024-09-11T00:20:24.021908+0000","last_unstale":"2024-09-11T00:23:28.401683+0000","last_undegraded":"2024-09-11T00:23:28.401683+0000","last_fullsized":"2024-09-11T00:23:28.401683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:35.688767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680798+0000","last_change":"2024-09-11T00:20:37.444898+0000","last_active":"2024-09-11T00:22:28.680798+0000","last_peered":"2024-09-11T00:22:28.680798+0000","last_clean":"2024-09-11T00:22:28.680798+0000","last_became_active":"2024-09-11T00:20:37.443840+0000","last_became_peered":"2024-09-11T00:20:37.443840+0000","last_unstale":"2024-09-11T00:22:28.680798+0000","last_undegraded":"2024-09-11T00:22:28.680798+0000","last_fullsized":"2024-09-11T00:22:28.680798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:58.668882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606050+0000","last_change":"2024-09-11T00:20:27.061198+0000","last_active":"2024-09-11T00:22:56.606050+0000","last_peered":"2024-09-11T00:22:56.606050+0000","last_clean":"2024-09-11T00:22:56.606050+0000","last_became_active":"2024-09-11T00:20:27.060877+0000","last_became_peered":"2024-09-11T00:20:27.060877+0000","last_unstale":"2024-09-11T00:22:56.606050+0000","last_undegraded":"2024-09-11T00:22:56.606050+0000","last_fullsized":"2024-09-11T00:22:56.606050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:03.595262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734264+0000","last_change":"2024-09-11T00:20:24.022755+0000","last_active":"2024-09-11T00:22:28.734264+0000","last_peered":"2024-09-11T00:22:28.734264+0000","last_clean":"2024-09-11T00:22:28.734264+0000","last_became_active":"2024-09-11T00:20:24.021530+0000","last_became_peered":"2024-09-11T00:20:24.021530+0000","last_unstale":"2024-09-11T00:22:28.734264+0000","last_undegraded":"2024-09-11T00:22:28.734264+0000","last_fullsized":"2024-09-11T00:22:28.734264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:33.881000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864371+0000","last_change":"2024-09-11T00:20:37.443722+0000","last_active":"2024-09-11T00:22:28.864371+0000","last_peered":"2024-09-11T00:22:28.864371+0000","last_clean":"2024-09-11T00:22:28.864371+0000","last_became_active":"2024-09-11T00:20:37.443339+0000","last_became_peered":"2024-09-11T00:20:37.443339+0000","last_unstale":"2024-09-11T00:22:28.864371+0000","last_undegraded":"2024-09-11T00:22:28.864371+0000","last_fullsized":"2024-09-11T00:22:28.864371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:30.221804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610517+0000","last_change":"2024-09-11T00:20:27.073158+0000","last_active":"2024-09-11T00:22:56.610517+0000","last_peered":"2024-09-11T00:22:56.610517+0000","last_clean":"2024-09-11T00:22:56.610517+0000","last_became_active":"2024-09-11T00:20:27.072805+0000","last_became_peered":"2024-09-11T00:20:27.072805+0000","last_unstale":"2024-09-11T00:22:56.610517+0000","last_undegraded":"2024-09-11T00:22:56.610517+0000","last_fullsized":"2024-09-11T00:22:56.610517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:35.124632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679703+0000","last_change":"2024-09-11T00:20:24.027178+0000","last_active":"2024-09-11T00:22:28.679703+0000","last_peered":"2024-09-11T00:22:28.679703+0000","last_clean":"2024-09-11T00:22:28.679703+0000","last_became_active":"2024-09-11T00:20:24.026821+0000","last_became_peered":"2024-09-11T00:20:24.026821+0000","last_unstale":"2024-09-11T00:22:28.679703+0000","last_undegraded":"2024-09-11T00:22:28.679703+0000","last_fullsized":"2024-09-11T00:22:28.679703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:38.393135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734422+0000","last_change":"2024-09-11T00:20:37.447587+0000","last_active":"2024-09-11T00:22:28.734422+0000","last_peered":"2024-09-11T00:22:28.734422+0000","last_clean":"2024-09-11T00:22:28.734422+0000","last_became_active":"2024-09-11T00:20:37.447166+0000","last_became_peered":"2024-09-11T00:20:37.447166+0000","last_unstale":"2024-09-11T00:22:28.734422+0000","last_undegraded":"2024-09-11T00:22:28.734422+0000","last_fullsized":"2024-09-11T00:22:28.734422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:47.469539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605796+0000","last_change":"2024-09-11T00:20:27.059914+0000","last_active":"2024-09-11T00:22:56.605796+0000","last_peered":"2024-09-11T00:22:56.605796+0000","last_clean":"2024-09-11T00:22:56.605796+0000","last_became_active":"2024-09-11T00:20:27.059595+0000","last_became_peered":"2024-09-11T00:20:27.059595+0000","last_unstale":"2024-09-11T00:22:56.605796+0000","last_undegraded":"2024-09-11T00:22:56.605796+0000","last_fullsized":"2024-09-11T00:22:56.605796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.360010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":343,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:28.424758+0000","last_change":"2024-09-11T00:20:24.023094+0000","last_active":"2024-09-11T00:23:28.424758+0000","last_peered":"2024-09-11T00:23:28.424758+0000","last_clean":"2024-09-11T00:23:28.424758+0000","last_became_active":"2024-09-11T00:20:24.022742+0000","last_became_peered":"2024-09-11T00:20:24.022742+0000","last_unstale":"2024-09-11T00:23:28.424758+0000","last_undegraded":"2024-09-11T00:23:28.424758+0000","last_fullsized":"2024-09-11T00:23:28.424758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:41.354435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62632,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":191,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680599+0000","last_change":"2024-09-11T00:20:37.451610+0000","last_active":"2024-09-11T00:22:28.680599+0000","last_peered":"2024-09-11T00:22:28.680599+0000","last_clean":"2024-09-11T00:22:28.680599+0000","last_became_active":"2024-09-11T00:20:37.444233+0000","last_became_peered":"2024-09-11T00:20:37.444233+0000","last_unstale":"2024-09-11T00:22:28.680599+0000","last_undegraded":"2024-09-11T00:22:28.680599+0000","last_fullsized":"2024-09-11T00:22:28.680599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:02.558457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608959+0000","last_change":"2024-09-11T00:20:27.061229+0000","last_active":"2024-09-11T00:22:56.608959+0000","last_peered":"2024-09-11T00:22:56.608959+0000","last_clean":"2024-09-11T00:22:56.608959+0000","last_became_active":"2024-09-11T00:20:27.060360+0000","last_became_peered":"2024-09-11T00:20:27.060360+0000","last_unstale":"2024-09-11T00:22:56.608959+0000","last_undegraded":"2024-09-11T00:22:56.608959+0000","last_fullsized":"2024-09-11T00:22:56.608959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.107405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734591+0000","last_change":"2024-09-11T00:20:24.036483+0000","last_active":"2024-09-11T00:22:28.734591+0000","last_peered":"2024-09-11T00:22:28.734591+0000","last_clean":"2024-09-11T00:22:28.734591+0000","last_became_active":"2024-09-11T00:20:24.035855+0000","last_became_peered":"2024-09-11T00:20:24.035855+0000","last_unstale":"2024-09-11T00:22:28.734591+0000","last_undegraded":"2024-09-11T00:22:28.734591+0000","last_fullsized":"2024-09-11T00:22:28.734591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.611169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864762+0000","last_change":"2024-09-11T00:21:28.245665+0000","last_active":"2024-09-11T00:22:28.864762+0000","last_peered":"2024-09-11T00:22:28.864762+0000","last_clean":"2024-09-11T00:22:28.864762+0000","last_became_active":"2024-09-11T00:21:28.245221+0000","last_became_peered":"2024-09-11T00:21:28.245221+0000","last_unstale":"2024-09-11T00:22:28.864762+0000","last_undegraded":"2024-09-11T00:22:28.864762+0000","last_fullsized":"2024-09-11T00:22:28.864762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:36.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611454+0000","last_change":"2024-09-11T00:21:28.231435+0000","last_active":"2024-09-11T00:22:56.611454+0000","last_peered":"2024-09-11T00:22:56.611454+0000","last_clean":"2024-09-11T00:22:56.611454+0000","last_became_active":"2024-09-11T00:21:28.230956+0000","last_became_peered":"2024-09-11T00:21:28.230956+0000","last_unstale":"2024-09-11T00:22:56.611454+0000","last_undegraded":"2024-09-11T00:22:56.611454+0000","last_fullsized":"2024-09-11T00:22:56.611454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:28.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865542+0000","last_change":"2024-09-11T00:20:24.023068+0000","last_active":"2024-09-11T00:22:28.865542+0000","last_peered":"2024-09-11T00:22:28.865542+0000","last_clean":"2024-09-11T00:22:28.865542+0000","last_became_active":"2024-09-11T00:20:24.022718+0000","last_became_peered":"2024-09-11T00:20:24.022718+0000","last_unstale":"2024-09-11T00:22:28.865542+0000","last_undegraded":"2024-09-11T00:22:28.865542+0000","last_fullsized":"2024-09-11T00:22:28.865542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.651536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863633+0000","last_change":"2024-09-11T00:21:28.231887+0000","last_active":"2024-09-11T00:22:28.863633+0000","last_peered":"2024-09-11T00:22:28.863633+0000","last_clean":"2024-09-11T00:22:28.863633+0000","last_became_active":"2024-09-11T00:21:28.231456+0000","last_became_peered":"2024-09-11T00:21:28.231456+0000","last_unstale":"2024-09-11T00:22:28.863633+0000","last_undegraded":"2024-09-11T00:22:28.863633+0000","last_fullsized":"2024-09-11T00:22:28.863633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:32.712598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613303+0000","last_change":"2024-09-11T00:20:27.065109+0000","last_active":"2024-09-11T00:22:56.613303+0000","last_peered":"2024-09-11T00:22:56.613303+0000","last_clean":"2024-09-11T00:22:56.613303+0000","last_became_active":"2024-09-11T00:20:27.064677+0000","last_became_peered":"2024-09-11T00:20:27.064677+0000","last_unstale":"2024-09-11T00:22:56.613303+0000","last_undegraded":"2024-09-11T00:22:56.613303+0000","last_fullsized":"2024-09-11T00:22:56.613303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:19.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:27.399849+0000","last_change":"2024-09-11T00:20:24.025347+0000","last_active":"2024-09-11T00:23:27.399849+0000","last_peered":"2024-09-11T00:23:27.399849+0000","last_clean":"2024-09-11T00:23:27.399849+0000","last_became_active":"2024-09-11T00:20:24.024980+0000","last_became_peered":"2024-09-11T00:20:24.024980+0000","last_unstale":"2024-09-11T00:23:27.399849+0000","last_undegraded":"2024-09-11T00:23:27.399849+0000","last_fullsized":"2024-09-11T00:23:27.399849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:12.530339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863597+0000","last_change":"2024-09-11T00:20:37.440340+0000","last_active":"2024-09-11T00:22:28.863597+0000","last_peered":"2024-09-11T00:22:28.863597+0000","last_clean":"2024-09-11T00:22:28.863597+0000","last_became_active":"2024-09-11T00:20:37.439896+0000","last_became_peered":"2024-09-11T00:20:37.439896+0000","last_unstale":"2024-09-11T00:22:28.863597+0000","last_undegraded":"2024-09-11T00:22:28.863597+0000","last_fullsized":"2024-09-11T00:22:28.863597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:38.774065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679198+0000","last_change":"2024-09-11T00:20:37.445860+0000","last_active":"2024-09-11T00:22:28.679198+0000","last_peered":"2024-09-11T00:22:28.679198+0000","last_clean":"2024-09-11T00:22:28.679198+0000","last_became_active":"2024-09-11T00:20:37.445450+0000","last_became_peered":"2024-09-11T00:20:37.445450+0000","last_unstale":"2024-09-11T00:22:28.679198+0000","last_undegraded":"2024-09-11T00:22:28.679198+0000","last_fullsized":"2024-09-11T00:22:28.679198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:44.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607656+0000","last_change":"2024-09-11T00:20:27.073529+0000","last_active":"2024-09-11T00:22:56.607656+0000","last_peered":"2024-09-11T00:22:56.607656+0000","last_clean":"2024-09-11T00:22:56.607656+0000","last_became_active":"2024-09-11T00:20:27.073168+0000","last_became_peered":"2024-09-11T00:20:27.073168+0000","last_unstale":"2024-09-11T00:22:56.607656+0000","last_undegraded":"2024-09-11T00:22:56.607656+0000","last_fullsized":"2024-09-11T00:22:56.607656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:16.407543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734124+0000","last_change":"2024-09-11T00:20:24.021778+0000","last_active":"2024-09-11T00:22:28.734124+0000","last_peered":"2024-09-11T00:22:28.734124+0000","last_clean":"2024-09-11T00:22:28.734124+0000","last_became_active":"2024-09-11T00:20:24.021443+0000","last_became_peered":"2024-09-11T00:20:24.021443+0000","last_unstale":"2024-09-11T00:22:28.734124+0000","last_undegraded":"2024-09-11T00:22:28.734124+0000","last_fullsized":"2024-09-11T00:22:28.734124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864397+0000","last_change":"2024-09-11T00:20:37.444279+0000","last_active":"2024-09-11T00:22:28.864397+0000","last_peered":"2024-09-11T00:22:28.864397+0000","last_clean":"2024-09-11T00:22:28.864397+0000","last_became_active":"2024-09-11T00:20:37.443821+0000","last_became_peered":"2024-09-11T00:20:37.443821+0000","last_unstale":"2024-09-11T00:22:28.864397+0000","last_undegraded":"2024-09-11T00:22:28.864397+0000","last_fullsized":"2024-09-11T00:22:28.864397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:10.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606265+0000","last_change":"2024-09-11T00:20:27.066155+0000","last_active":"2024-09-11T00:22:56.606265+0000","last_peered":"2024-09-11T00:22:56.606265+0000","last_clean":"2024-09-11T00:22:56.606265+0000","last_became_active":"2024-09-11T00:20:27.057262+0000","last_became_peered":"2024-09-11T00:20:27.057262+0000","last_unstale":"2024-09-11T00:22:56.606265+0000","last_undegraded":"2024-09-11T00:22:56.606265+0000","last_fullsized":"2024-09-11T00:22:56.606265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:54.160215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682647+0000","last_change":"2024-09-11T00:20:24.026123+0000","last_active":"2024-09-11T00:22:28.682647+0000","last_peered":"2024-09-11T00:22:28.682647+0000","last_clean":"2024-09-11T00:22:28.682647+0000","last_became_active":"2024-09-11T00:20:24.025832+0000","last_became_peered":"2024-09-11T00:20:24.025832+0000","last_unstale":"2024-09-11T00:22:28.682647+0000","last_undegraded":"2024-09-11T00:22:28.682647+0000","last_fullsized":"2024-09-11T00:22:28.682647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:45.444840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830187+0000","last_change":"2024-09-11T00:20:24.028523+0000","last_active":"2024-09-11T00:22:28.830187+0000","last_peered":"2024-09-11T00:22:28.830187+0000","last_clean":"2024-09-11T00:22:28.830187+0000","last_became_active":"2024-09-11T00:20:24.027801+0000","last_became_peered":"2024-09-11T00:20:24.027801+0000","last_unstale":"2024-09-11T00:22:28.830187+0000","last_undegraded":"2024-09-11T00:22:28.830187+0000","last_fullsized":"2024-09-11T00:22:28.830187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:14.174983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607124+0000","last_change":"2024-09-11T00:20:27.074634+0000","last_active":"2024-09-11T00:22:56.607124+0000","last_peered":"2024-09-11T00:22:56.607124+0000","last_clean":"2024-09-11T00:22:56.607124+0000","last_became_active":"2024-09-11T00:20:27.074292+0000","last_became_peered":"2024-09-11T00:20:27.074292+0000","last_unstale":"2024-09-11T00:22:56.607124+0000","last_undegraded":"2024-09-11T00:22:56.607124+0000","last_fullsized":"2024-09-11T00:22:56.607124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:07.467262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679446+0000","last_change":"2024-09-11T00:20:37.454341+0000","last_active":"2024-09-11T00:22:28.679446+0000","last_peered":"2024-09-11T00:22:28.679446+0000","last_clean":"2024-09-11T00:22:28.679446+0000","last_became_active":"2024-09-11T00:20:37.453868+0000","last_became_peered":"2024-09-11T00:20:37.453868+0000","last_unstale":"2024-09-11T00:22:28.679446+0000","last_undegraded":"2024-09-11T00:22:28.679446+0000","last_fullsized":"2024-09-11T00:22:28.679446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.015338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:27.400519+0000","last_change":"2024-09-11T00:20:24.025184+0000","last_active":"2024-09-11T00:23:27.400519+0000","last_peered":"2024-09-11T00:23:27.400519+0000","last_clean":"2024-09-11T00:23:27.400519+0000","last_became_active":"2024-09-11T00:20:24.024670+0000","last_became_peered":"2024-09-11T00:20:24.024670+0000","last_unstale":"2024-09-11T00:23:27.400519+0000","last_undegraded":"2024-09-11T00:23:27.400519+0000","last_fullsized":"2024-09-11T00:23:27.400519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:40.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594092+0000","last_change":"2024-09-11T00:20:27.057822+0000","last_active":"2024-09-11T00:22:56.594092+0000","last_peered":"2024-09-11T00:22:56.594092+0000","last_clean":"2024-09-11T00:22:56.594092+0000","last_became_active":"2024-09-11T00:20:27.057518+0000","last_became_peered":"2024-09-11T00:20:27.057518+0000","last_unstale":"2024-09-11T00:22:56.594092+0000","last_undegraded":"2024-09-11T00:22:56.594092+0000","last_fullsized":"2024-09-11T00:22:56.594092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:00.264327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679539+0000","last_change":"2024-09-11T00:20:37.438369+0000","last_active":"2024-09-11T00:22:28.679539+0000","last_peered":"2024-09-11T00:22:28.679539+0000","last_clean":"2024-09-11T00:22:28.679539+0000","last_became_active":"2024-09-11T00:20:37.437942+0000","last_became_peered":"2024-09-11T00:20:37.437942+0000","last_unstale":"2024-09-11T00:22:28.679539+0000","last_undegraded":"2024-09-11T00:22:28.679539+0000","last_fullsized":"2024-09-11T00:22:28.679539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.410905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.595081+0000","last_change":"2024-09-11T00:20:27.057449+0000","last_active":"2024-09-11T00:22:56.595081+0000","last_peered":"2024-09-11T00:22:56.595081+0000","last_clean":"2024-09-11T00:22:56.595081+0000","last_became_active":"2024-09-11T00:20:27.057146+0000","last_became_peered":"2024-09-11T00:20:27.057146+0000","last_unstale":"2024-09-11T00:22:56.595081+0000","last_undegraded":"2024-09-11T00:22:56.595081+0000","last_fullsized":"2024-09-11T00:22:56.595081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:57.895062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.603116+0000","last_change":"2024-09-11T00:20:27.061515+0000","last_active":"2024-09-11T00:22:56.603116+0000","last_peered":"2024-09-11T00:22:56.603116+0000","last_clean":"2024-09-11T00:22:56.603116+0000","last_became_active":"2024-09-11T00:20:27.061175+0000","last_became_peered":"2024-09-11T00:20:27.061175+0000","last_unstale":"2024-09-11T00:22:56.603116+0000","last_undegraded":"2024-09-11T00:22:56.603116+0000","last_fullsized":"2024-09-11T00:22:56.603116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:03.112574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.609556+0000","last_change":"2024-09-11T00:20:27.063015+0000","last_active":"2024-09-11T00:22:56.609556+0000","last_peered":"2024-09-11T00:22:56.609556+0000","last_clean":"2024-09-11T00:22:56.609556+0000","last_became_active":"2024-09-11T00:20:27.062689+0000","last_became_peered":"2024-09-11T00:20:27.062689+0000","last_unstale":"2024-09-11T00:22:56.609556+0000","last_undegraded":"2024-09-11T00:22:56.609556+0000","last_fullsized":"2024-09-11T00:22:56.609556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.736073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605982+0000","last_change":"2024-09-11T00:20:27.062196+0000","last_active":"2024-09-11T00:22:56.605982+0000","last_peered":"2024-09-11T00:22:56.605982+0000","last_clean":"2024-09-11T00:22:56.605982+0000","last_became_active":"2024-09-11T00:20:27.061918+0000","last_became_peered":"2024-09-11T00:20:27.061918+0000","last_unstale":"2024-09-11T00:22:56.605982+0000","last_undegraded":"2024-09-11T00:22:56.605982+0000","last_fullsized":"2024-09-11T00:22:56.605982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:45.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611720+0000","last_change":"2024-09-11T00:20:27.059021+0000","last_active":"2024-09-11T00:22:56.611720+0000","last_peered":"2024-09-11T00:22:56.611720+0000","last_clean":"2024-09-11T00:22:56.611720+0000","last_became_active":"2024-09-11T00:20:27.058726+0000","last_became_peered":"2024-09-11T00:20:27.058726+0000","last_unstale":"2024-09-11T00:22:56.611720+0000","last_undegraded":"2024-09-11T00:22:56.611720+0000","last_fullsized":"2024-09-11T00:22:56.611720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:23.900236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.593828+0000","last_change":"2024-09-11T00:20:27.057593+0000","last_active":"2024-09-11T00:22:56.593828+0000","last_peered":"2024-09-11T00:22:56.593828+0000","last_clean":"2024-09-11T00:22:56.593828+0000","last_became_active":"2024-09-11T00:20:27.057310+0000","last_became_peered":"2024-09-11T00:20:27.057310+0000","last_unstale":"2024-09-11T00:22:56.593828+0000","last_undegraded":"2024-09-11T00:22:56.593828+0000","last_fullsized":"2024-09-11T00:22:56.593828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:52.355218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605460+0000","last_change":"2024-09-11T00:20:27.061140+0000","last_active":"2024-09-11T00:22:56.605460+0000","last_peered":"2024-09-11T00:22:56.605460+0000","last_clean":"2024-09-11T00:22:56.605460+0000","last_became_active":"2024-09-11T00:20:27.060500+0000","last_became_peered":"2024-09-11T00:20:27.060500+0000","last_unstale":"2024-09-11T00:22:56.605460+0000","last_undegraded":"2024-09-11T00:22:56.605460+0000","last_fullsized":"2024-09-11T00:22:56.605460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.159068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.591859+0000","last_change":"2024-09-11T00:20:27.061936+0000","last_active":"2024-09-11T00:22:56.591859+0000","last_peered":"2024-09-11T00:22:56.591859+0000","last_clean":"2024-09-11T00:22:56.591859+0000","last_became_active":"2024-09-11T00:20:27.061616+0000","last_became_peered":"2024-09-11T00:20:27.061616+0000","last_unstale":"2024-09-11T00:22:56.591859+0000","last_undegraded":"2024-09-11T00:22:56.591859+0000","last_fullsized":"2024-09-11T00:22:56.591859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:32.185634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608892+0000","last_change":"2024-09-11T00:20:27.074645+0000","last_active":"2024-09-11T00:22:56.608892+0000","last_peered":"2024-09-11T00:22:56.608892+0000","last_clean":"2024-09-11T00:22:56.608892+0000","last_became_active":"2024-09-11T00:20:27.074355+0000","last_became_peered":"2024-09-11T00:20:27.074355+0000","last_unstale":"2024-09-11T00:22:56.608892+0000","last_undegraded":"2024-09-11T00:22:56.608892+0000","last_fullsized":"2024-09-11T00:22:56.608892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.601812+0000","last_change":"2024-09-11T00:20:27.069588+0000","last_active":"2024-09-11T00:22:56.601812+0000","last_peered":"2024-09-11T00:22:56.601812+0000","last_clean":"2024-09-11T00:22:56.601812+0000","last_became_active":"2024-09-11T00:20:27.067403+0000","last_became_peered":"2024-09-11T00:20:27.067403+0000","last_unstale":"2024-09-11T00:22:56.601812+0000","last_undegraded":"2024-09-11T00:22:56.601812+0000","last_fullsized":"2024-09-11T00:22:56.601812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:29.439013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610162+0000","last_change":"2024-09-11T00:20:27.071586+0000","last_active":"2024-09-11T00:22:56.610162+0000","last_peered":"2024-09-11T00:22:56.610162+0000","last_clean":"2024-09-11T00:22:56.610162+0000","last_became_active":"2024-09-11T00:20:27.071072+0000","last_became_peered":"2024-09-11T00:20:27.071072+0000","last_unstale":"2024-09-11T00:22:56.610162+0000","last_undegraded":"2024-09-11T00:22:56.610162+0000","last_fullsized":"2024-09-11T00:22:56.610162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:34.133027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613642+0000","last_change":"2024-09-11T00:20:27.066683+0000","last_active":"2024-09-11T00:22:56.613642+0000","last_peered":"2024-09-11T00:22:56.613642+0000","last_clean":"2024-09-11T00:22:56.613642+0000","last_became_active":"2024-09-11T00:20:27.066355+0000","last_became_peered":"2024-09-11T00:20:27.066355+0000","last_unstale":"2024-09-11T00:22:56.613642+0000","last_undegraded":"2024-09-11T00:22:56.613642+0000","last_fullsized":"2024-09-11T00:22:56.613642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:26.868072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.598228+0000","last_change":"2024-09-11T00:20:27.060546+0000","last_active":"2024-09-11T00:22:56.598228+0000","last_peered":"2024-09-11T00:22:56.598228+0000","last_clean":"2024-09-11T00:22:56.598228+0000","last_became_active":"2024-09-11T00:20:27.059619+0000","last_became_peered":"2024-09-11T00:20:27.059619+0000","last_unstale":"2024-09-11T00:22:56.598228+0000","last_undegraded":"2024-09-11T00:22:56.598228+0000","last_fullsized":"2024-09-11T00:22:56.598228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.319854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607755+0000","last_change":"2024-09-11T00:20:27.073342+0000","last_active":"2024-09-11T00:22:56.607755+0000","last_peered":"2024-09-11T00:22:56.607755+0000","last_clean":"2024-09-11T00:22:56.607755+0000","last_became_active":"2024-09-11T00:20:27.072435+0000","last_became_peered":"2024-09-11T00:20:27.072435+0000","last_unstale":"2024-09-11T00:22:56.607755+0000","last_undegraded":"2024-09-11T00:22:56.607755+0000","last_fullsized":"2024-09-11T00:22:56.607755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:15.943515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612461+0000","last_change":"2024-09-11T00:20:27.063220+0000","last_active":"2024-09-11T00:22:56.612461+0000","last_peered":"2024-09-11T00:22:56.612461+0000","last_clean":"2024-09-11T00:22:56.612461+0000","last_became_active":"2024-09-11T00:20:27.062823+0000","last_became_peered":"2024-09-11T00:20:27.062823+0000","last_unstale":"2024-09-11T00:22:56.612461+0000","last_undegraded":"2024-09-11T00:22:56.612461+0000","last_fullsized":"2024-09-11T00:22:56.612461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:22.430295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613071+0000","last_change":"2024-09-11T00:20:27.062088+0000","last_active":"2024-09-11T00:22:56.613071+0000","last_peered":"2024-09-11T00:22:56.613071+0000","last_clean":"2024-09-11T00:22:56.613071+0000","last_became_active":"2024-09-11T00:20:27.061639+0000","last_became_peered":"2024-09-11T00:20:27.061639+0000","last_unstale":"2024-09-11T00:22:56.613071+0000","last_undegraded":"2024-09-11T00:22:56.613071+0000","last_fullsized":"2024-09-11T00:22:56.613071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.965908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611123+0000","last_change":"2024-09-11T00:20:27.067170+0000","last_active":"2024-09-11T00:22:56.611123+0000","last_peered":"2024-09-11T00:22:56.611123+0000","last_clean":"2024-09-11T00:22:56.611123+0000","last_became_active":"2024-09-11T00:20:27.066851+0000","last_became_peered":"2024-09-11T00:20:27.066851+0000","last_unstale":"2024-09-11T00:22:56.611123+0000","last_undegraded":"2024-09-11T00:22:56.611123+0000","last_fullsized":"2024-09-11T00:22:56.611123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:51.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604750+0000","last_change":"2024-09-11T00:21:28.233390+0000","last_active":"2024-09-11T00:22:56.604750+0000","last_peered":"2024-09-11T00:22:56.604750+0000","last_clean":"2024-09-11T00:22:56.604750+0000","last_became_active":"2024-09-11T00:21:28.232886+0000","last_became_peered":"2024-09-11T00:21:28.232886+0000","last_unstale":"2024-09-11T00:22:56.604750+0000","last_undegraded":"2024-09-11T00:22:56.604750+0000","last_fullsized":"2024-09-11T00:22:56.604750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:26.702069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606581+0000","last_change":"2024-09-11T00:20:27.059553+0000","last_active":"2024-09-11T00:22:56.606581+0000","last_peered":"2024-09-11T00:22:56.606581+0000","last_clean":"2024-09-11T00:22:56.606581+0000","last_became_active":"2024-09-11T00:20:27.058979+0000","last_became_peered":"2024-09-11T00:20:27.058979+0000","last_unstale":"2024-09-11T00:22:56.606581+0000","last_undegraded":"2024-09-11T00:22:56.606581+0000","last_fullsized":"2024-09-11T00:22:56.606581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:21.472828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606329+0000","last_change":"2024-09-11T00:22:28.831536+0000","last_active":"2024-09-11T00:22:56.606329+0000","last_peered":"2024-09-11T00:22:56.606329+0000","last_clean":"2024-09-11T00:22:56.606329+0000","last_became_active":"2024-09-11T00:22:28.831151+0000","last_became_peered":"2024-09-11T00:22:28.831151+0000","last_unstale":"2024-09-11T00:22:56.606329+0000","last_undegraded":"2024-09-11T00:22:56.606329+0000","last_fullsized":"2024-09-11T00:22:56.606329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.532570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.602775+0000","last_change":"2024-09-11T00:20:27.059140+0000","last_active":"2024-09-11T00:22:56.602775+0000","last_peered":"2024-09-11T00:22:56.602775+0000","last_clean":"2024-09-11T00:22:56.602775+0000","last_became_active":"2024-09-11T00:20:27.058765+0000","last_became_peered":"2024-09-11T00:20:27.058765+0000","last_unstale":"2024-09-11T00:22:56.602775+0000","last_undegraded":"2024-09-11T00:22:56.602775+0000","last_fullsized":"2024-09-11T00:22:56.602775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:43.947411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604652+0000","last_change":"2024-09-11T00:20:27.061149+0000","last_active":"2024-09-11T00:22:56.604652+0000","last_peered":"2024-09-11T00:22:56.604652+0000","last_clean":"2024-09-11T00:22:56.604652+0000","last_became_active":"2024-09-11T00:20:27.059949+0000","last_became_peered":"2024-09-11T00:20:27.059949+0000","last_unstale":"2024-09-11T00:22:56.604652+0000","last_undegraded":"2024-09-11T00:22:56.604652+0000","last_fullsized":"2024-09-11T00:22:56.604652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:03.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597666+0000","last_change":"2024-09-11T00:21:28.231322+0000","last_active":"2024-09-11T00:22:56.597666+0000","last_peered":"2024-09-11T00:22:56.597666+0000","last_clean":"2024-09-11T00:22:56.597666+0000","last_became_active":"2024-09-11T00:21:28.231075+0000","last_became_peered":"2024-09-11T00:21:28.231075+0000","last_unstale":"2024-09-11T00:22:56.597666+0000","last_undegraded":"2024-09-11T00:22:56.597666+0000","last_fullsized":"2024-09-11T00:22:56.597666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:50.887001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612406+0000","last_change":"2024-09-11T00:20:27.058175+0000","last_active":"2024-09-11T00:22:56.612406+0000","last_peered":"2024-09-11T00:22:56.612406+0000","last_clean":"2024-09-11T00:22:56.612406+0000","last_became_active":"2024-09-11T00:20:27.057797+0000","last_became_peered":"2024-09-11T00:20:27.057797+0000","last_unstale":"2024-09-11T00:22:56.612406+0000","last_undegraded":"2024-09-11T00:22:56.612406+0000","last_fullsized":"2024-09-11T00:22:56.612406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:29.370400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.609823+0000","last_change":"2024-09-11T00:20:27.062024+0000","last_active":"2024-09-11T00:22:56.609823+0000","last_peered":"2024-09-11T00:22:56.609823+0000","last_clean":"2024-09-11T00:22:56.609823+0000","last_became_active":"2024-09-11T00:20:27.061763+0000","last_became_peered":"2024-09-11T00:20:27.061763+0000","last_unstale":"2024-09-11T00:22:56.609823+0000","last_undegraded":"2024-09-11T00:22:56.609823+0000","last_fullsized":"2024-09-11T00:22:56.609823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:07.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597966+0000","last_change":"2024-09-11T00:20:27.060258+0000","last_active":"2024-09-11T00:22:56.597966+0000","last_peered":"2024-09-11T00:22:56.597966+0000","last_clean":"2024-09-11T00:22:56.597966+0000","last_became_active":"2024-09-11T00:20:27.059475+0000","last_became_peered":"2024-09-11T00:20:27.059475+0000","last_unstale":"2024-09-11T00:22:56.597966+0000","last_undegraded":"2024-09-11T00:22:56.597966+0000","last_fullsized":"2024-09-11T00:22:56.597966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:27.186322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.603092+0000","last_change":"2024-09-11T00:20:27.059730+0000","last_active":"2024-09-11T00:22:56.603092+0000","last_peered":"2024-09-11T00:22:56.603092+0000","last_clean":"2024-09-11T00:22:56.603092+0000","last_became_active":"2024-09-11T00:20:27.059389+0000","last_became_peered":"2024-09-11T00:20:27.059389+0000","last_unstale":"2024-09-11T00:22:56.603092+0000","last_undegraded":"2024-09-11T00:22:56.603092+0000","last_fullsized":"2024-09-11T00:22:56.603092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605141+0000","last_change":"2024-09-11T00:20:27.072765+0000","last_active":"2024-09-11T00:22:56.605141+0000","last_peered":"2024-09-11T00:22:56.605141+0000","last_clean":"2024-09-11T00:22:56.605141+0000","last_became_active":"2024-09-11T00:20:27.072392+0000","last_became_peered":"2024-09-11T00:20:27.072392+0000","last_unstale":"2024-09-11T00:22:56.605141+0000","last_undegraded":"2024-09-11T00:22:56.605141+0000","last_fullsized":"2024-09-11T00:22:56.605141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:27.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.600287+0000","last_change":"2024-09-11T00:20:27.056575+0000","last_active":"2024-09-11T00:22:56.600287+0000","last_peered":"2024-09-11T00:22:56.600287+0000","last_clean":"2024-09-11T00:22:56.600287+0000","last_became_active":"2024-09-11T00:20:27.056264+0000","last_became_peered":"2024-09-11T00:20:27.056264+0000","last_unstale":"2024-09-11T00:22:56.600287+0000","last_undegraded":"2024-09-11T00:22:56.600287+0000","last_fullsized":"2024-09-11T00:22:56.600287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:39.722518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610079+0000","last_change":"2024-09-11T00:20:27.065153+0000","last_active":"2024-09-11T00:22:56.610079+0000","last_peered":"2024-09-11T00:22:56.610079+0000","last_clean":"2024-09-11T00:22:56.610079+0000","last_became_active":"2024-09-11T00:20:27.062978+0000","last_became_peered":"2024-09-11T00:20:27.062978+0000","last_unstale":"2024-09-11T00:22:56.610079+0000","last_undegraded":"2024-09-11T00:22:56.610079+0000","last_fullsized":"2024-09-11T00:22:56.610079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:37.820467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605823+0000","last_change":"2024-09-11T00:20:27.070085+0000","last_active":"2024-09-11T00:22:56.605823+0000","last_peered":"2024-09-11T00:22:56.605823+0000","last_clean":"2024-09-11T00:22:56.605823+0000","last_became_active":"2024-09-11T00:20:27.069809+0000","last_became_peered":"2024-09-11T00:20:27.069809+0000","last_unstale":"2024-09-11T00:22:56.605823+0000","last_undegraded":"2024-09-11T00:22:56.605823+0000","last_fullsized":"2024-09-11T00:22:56.605823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:01.048590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612973+0000","last_change":"2024-09-11T00:20:27.064584+0000","last_active":"2024-09-11T00:22:56.612973+0000","last_peered":"2024-09-11T00:22:56.612973+0000","last_clean":"2024-09-11T00:22:56.612973+0000","last_became_active":"2024-09-11T00:20:27.064271+0000","last_became_peered":"2024-09-11T00:20:27.064271+0000","last_unstale":"2024-09-11T00:22:56.612973+0000","last_undegraded":"2024-09-11T00:22:56.612973+0000","last_fullsized":"2024-09-11T00:22:56.612973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:53.717744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67085,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":772,"num_read_kb":456,"num_write":179,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710771,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":515118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906999,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1165217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070525,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266755,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":515050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462984,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1111523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":31914445},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626509,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":510418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822740,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":515379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":510612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":574531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":572649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":511806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:23:31.624 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:23:31.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:31 smithi183 ceph-mon[35934]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:31.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:31 smithi103 ceph-mon[30578]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:31 smithi117 ceph-mon[35525]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:31.998 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 -- 172.21.15.103:0/2545416958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda440776d0 msgr2=0x7fda44077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 --2- 172.21.15.103:0/2545416958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda440776d0 0x7fda44077ab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fda34009900 tx=0x7fda3402a410 comp rx=0 tx=0).stop 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 -- 172.21.15.103:0/2545416958 shutdown_connections 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 --2- 172.21.15.103:0/2545416958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda44113990 0x7fda44115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 --2- 172.21.15.103:0/2545416958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda44077ff0 0x7fda44113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 --2- 172.21.15.103:0/2545416958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda440776d0 0x7fda44077ab0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 -- 172.21.15.103:0/2545416958 >> 172.21.15.103:0/2545416958 conn(0x7fda44100a50 msgr2=0x7fda44102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 -- 172.21.15.103:0/2545416958 shutdown_connections 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.593+0000 7fda49a00640 1 -- 172.21.15.103:0/2545416958 wait complete. 2024-09-11T00:23:32.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.594+0000 7fda49a00640 1 Processor -- start 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda49a00640 1 -- start start 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda49a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda440776d0 0x7fda4419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda49a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda44077ff0 0x7fda4419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda49a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda44113990 0x7fda441a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda49a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda44119380 con 0x7fda44113990 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda49a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda44119200 con 0x7fda440776d0 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda49a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda44119500 con 0x7fda44077ff0 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.595+0000 7fda42a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda440776d0 0x7fda4419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda44077ff0 0x7fda4419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda43400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda44113990 0x7fda441a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda440776d0 0x7fda4419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:36938/0 (socket says 172.21.15.103:36938) 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 -- 172.21.15.103:0/818307473 learned_addr learned my addr 172.21.15.103:0/818307473 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:32.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 -- 172.21.15.103:0/818307473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda44077ff0 msgr2=0x7fda4419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda44077ff0 0x7fda4419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 -- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda44113990 msgr2=0x7fda441a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda44113990 0x7fda441a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 -- 172.21.15.103:0/818307473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda441a5960 con 0x7fda440776d0 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.596+0000 7fda42a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda440776d0 0x7fda4419f8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda34005940 tx=0x7fda340042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.597+0000 7fda3b400640 1 -- 172.21.15.103:0/818307473 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3403f030 con 0x7fda440776d0 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.597+0000 7fda3b400640 1 -- 172.21.15.103:0/818307473 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda34003cb0 con 0x7fda440776d0 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.597+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda441a5bf0 con 0x7fda440776d0 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.597+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441ac310 con 0x7fda440776d0 2024-09-11T00:23:32.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.597+0000 7fda3b400640 1 -- 172.21.15.103:0/818307473 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34035070 con 0x7fda440776d0 2024-09-11T00:23:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.598+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda44077ab0 con 0x7fda440776d0 2024-09-11T00:23:32.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.600+0000 7fda3b400640 1 -- 172.21.15.103:0/818307473 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fda3403bb70 con 0x7fda440776d0 2024-09-11T00:23:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.602+0000 7fda3b400640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda280b5360 0x7fda280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.603+0000 7fda3b400640 1 -- 172.21.15.103:0/818307473 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fda34106a10 con 0x7fda440776d0 2024-09-11T00:23:32.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.603+0000 7fda42000640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda280b5360 0x7fda280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:32.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.603+0000 7fda42000640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda280b5360 0x7fda280b7820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fda44116020 tx=0x7fda3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:32.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.606+0000 7fda3b400640 1 -- 172.21.15.103:0/818307473 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fda340cce90 con 0x7fda440776d0 2024-09-11T00:23:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:32 smithi183 ceph-mon[35934]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:32.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.794+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda44115640 con 0x7fda280b5360 2024-09-11T00:23:32.797 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:32 smithi103 ceph-mon[30578]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:32.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.800+0000 7fda3b400640 1 -- 172.21.15.103:0/818307473 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340474 (secure 0 0 0) 0x7fda44115640 con 0x7fda280b5360 2024-09-11T00:23:32.807 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:32.809 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-11T00:23:32.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda280b5360 msgr2=0x7fda280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:32.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda280b5360 0x7fda280b7820 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fda44116020 tx=0x7fda3000b390 comp rx=0 tx=0).stop 2024-09-11T00:23:32.811 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda440776d0 msgr2=0x7fda4419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda440776d0 0x7fda4419f8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda34005940 tx=0x7fda340042f0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 shutdown_connections 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda280b5360 0x7fda280b7820 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda44113990 0x7fda441a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda44077ff0 0x7fda4419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 --2- 172.21.15.103:0/818307473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda440776d0 0x7fda4419f8b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 >> 172.21.15.103:0/818307473 conn(0x7fda44100a50 msgr2=0x7fda44102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 shutdown_connections 2024-09-11T00:23:32.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:32.806+0000 7fda49a00640 1 -- 172.21.15.103:0/818307473 wait complete. 2024-09-11T00:23:32.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:32 smithi117 ceph-mon[35525]: pgmap v412: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:23:33.784 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-11T00:23:32.407783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657687,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":853,"num_read_kb":525,"num_write":307,"num_write_kb":2279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":749,"ondisk_log_size":749,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375828,"kb_used_data":14100,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565996,"statfs":{"total":1151940427776,"available":1151555579904,"internally_reserved":0,"allocated":14438400,"data_stored":8134125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":622,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":37,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003237"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680924+0000","last_change":"2024-09-11T00:20:24.027578+0000","last_active":"2024-09-11T00:22:28.680924+0000","last_peered":"2024-09-11T00:22:28.680924+0000","last_clean":"2024-09-11T00:22:28.680924+0000","last_became_active":"2024-09-11T00:20:24.027174+0000","last_became_peered":"2024-09-11T00:20:24.027174+0000","last_unstale":"2024-09-11T00:22:28.680924+0000","last_undegraded":"2024-09-11T00:22:28.680924+0000","last_fullsized":"2024-09-11T00:22:28.680924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:15.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.595270+0000","last_change":"2024-09-11T00:20:27.064047+0000","last_active":"2024-09-11T00:22:56.595270+0000","last_peered":"2024-09-11T00:22:56.595270+0000","last_clean":"2024-09-11T00:22:56.595270+0000","last_became_active":"2024-09-11T00:20:27.063756+0000","last_became_peered":"2024-09-11T00:20:27.063756+0000","last_unstale":"2024-09-11T00:22:56.595270+0000","last_undegraded":"2024-09-11T00:22:56.595270+0000","last_fullsized":"2024-09-11T00:22:56.595270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.695888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679261+0000","last_change":"2024-09-11T00:20:37.449428+0000","last_active":"2024-09-11T00:22:28.679261+0000","last_peered":"2024-09-11T00:22:28.679261+0000","last_clean":"2024-09-11T00:22:28.679261+0000","last_became_active":"2024-09-11T00:20:37.449015+0000","last_became_peered":"2024-09-11T00:20:37.449015+0000","last_unstale":"2024-09-11T00:22:28.679261+0000","last_undegraded":"2024-09-11T00:22:28.679261+0000","last_fullsized":"2024-09-11T00:22:28.679261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:42.983990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864076+0000","last_change":"2024-09-11T00:20:24.035966+0000","last_active":"2024-09-11T00:22:28.864076+0000","last_peered":"2024-09-11T00:22:28.864076+0000","last_clean":"2024-09-11T00:22:28.864076+0000","last_became_active":"2024-09-11T00:20:24.035663+0000","last_became_peered":"2024-09-11T00:20:24.035663+0000","last_unstale":"2024-09-11T00:22:28.864076+0000","last_undegraded":"2024-09-11T00:22:28.864076+0000","last_fullsized":"2024-09-11T00:22:28.864076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:22.529392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612592+0000","last_change":"2024-09-11T00:20:27.066470+0000","last_active":"2024-09-11T00:22:56.612592+0000","last_peered":"2024-09-11T00:22:56.612592+0000","last_clean":"2024-09-11T00:22:56.612592+0000","last_became_active":"2024-09-11T00:20:27.065424+0000","last_became_peered":"2024-09-11T00:20:27.065424+0000","last_unstale":"2024-09-11T00:22:56.612592+0000","last_undegraded":"2024-09-11T00:22:56.612592+0000","last_fullsized":"2024-09-11T00:22:56.612592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:39.967332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864816+0000","last_change":"2024-09-11T00:20:37.453105+0000","last_active":"2024-09-11T00:22:28.864816+0000","last_peered":"2024-09-11T00:22:28.864816+0000","last_clean":"2024-09-11T00:22:28.864816+0000","last_became_active":"2024-09-11T00:20:37.452319+0000","last_became_peered":"2024-09-11T00:20:37.452319+0000","last_unstale":"2024-09-11T00:22:28.864816+0000","last_undegraded":"2024-09-11T00:22:28.864816+0000","last_fullsized":"2024-09-11T00:22:28.864816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:05.873046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864807+0000","last_change":"2024-09-11T00:20:24.022277+0000","last_active":"2024-09-11T00:22:28.864807+0000","last_peered":"2024-09-11T00:22:28.864807+0000","last_clean":"2024-09-11T00:22:28.864807+0000","last_became_active":"2024-09-11T00:20:24.021893+0000","last_became_peered":"2024-09-11T00:20:24.021893+0000","last_unstale":"2024-09-11T00:22:28.864807+0000","last_undegraded":"2024-09-11T00:22:28.864807+0000","last_fullsized":"2024-09-11T00:22:28.864807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.994032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613024+0000","last_change":"2024-09-11T00:20:27.063368+0000","last_active":"2024-09-11T00:22:56.613024+0000","last_peered":"2024-09-11T00:22:56.613024+0000","last_clean":"2024-09-11T00:22:56.613024+0000","last_became_active":"2024-09-11T00:20:27.063051+0000","last_became_peered":"2024-09-11T00:20:27.063051+0000","last_unstale":"2024-09-11T00:22:56.613024+0000","last_undegraded":"2024-09-11T00:22:56.613024+0000","last_fullsized":"2024-09-11T00:22:56.613024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:09.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864842+0000","last_change":"2024-09-11T00:21:28.229073+0000","last_active":"2024-09-11T00:22:28.864842+0000","last_peered":"2024-09-11T00:22:28.864842+0000","last_clean":"2024-09-11T00:22:28.864842+0000","last_became_active":"2024-09-11T00:21:28.228775+0000","last_became_peered":"2024-09-11T00:21:28.228775+0000","last_unstale":"2024-09-11T00:22:28.864842+0000","last_undegraded":"2024-09-11T00:22:28.864842+0000","last_fullsized":"2024-09-11T00:22:28.864842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.886727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864759+0000","last_change":"2024-09-11T00:20:24.022562+0000","last_active":"2024-09-11T00:22:28.864759+0000","last_peered":"2024-09-11T00:22:28.864759+0000","last_clean":"2024-09-11T00:22:28.864759+0000","last_became_active":"2024-09-11T00:20:24.022207+0000","last_became_peered":"2024-09-11T00:20:24.022207+0000","last_unstale":"2024-09-11T00:22:28.864759+0000","last_undegraded":"2024-09-11T00:22:28.864759+0000","last_fullsized":"2024-09-11T00:22:28.864759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:01.071034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608323+0000","last_change":"2024-09-11T00:20:27.062572+0000","last_active":"2024-09-11T00:22:56.608323+0000","last_peered":"2024-09-11T00:22:56.608323+0000","last_clean":"2024-09-11T00:22:56.608323+0000","last_became_active":"2024-09-11T00:20:27.062288+0000","last_became_peered":"2024-09-11T00:20:27.062288+0000","last_unstale":"2024-09-11T00:22:56.608323+0000","last_undegraded":"2024-09-11T00:22:56.608323+0000","last_fullsized":"2024-09-11T00:22:56.608323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:37.391822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864680+0000","last_change":"2024-09-11T00:20:37.444028+0000","last_active":"2024-09-11T00:22:28.864680+0000","last_peered":"2024-09-11T00:22:28.864680+0000","last_clean":"2024-09-11T00:22:28.864680+0000","last_became_active":"2024-09-11T00:20:37.443344+0000","last_became_peered":"2024-09-11T00:20:37.443344+0000","last_unstale":"2024-09-11T00:22:28.864680+0000","last_undegraded":"2024-09-11T00:22:28.864680+0000","last_fullsized":"2024-09-11T00:22:28.864680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.347054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611869+0000","last_change":"2024-09-11T00:20:27.060317+0000","last_active":"2024-09-11T00:22:56.611869+0000","last_peered":"2024-09-11T00:22:56.611869+0000","last_clean":"2024-09-11T00:22:56.611869+0000","last_became_active":"2024-09-11T00:20:27.059983+0000","last_became_peered":"2024-09-11T00:20:27.059983+0000","last_unstale":"2024-09-11T00:22:56.611869+0000","last_undegraded":"2024-09-11T00:22:56.611869+0000","last_fullsized":"2024-09-11T00:22:56.611869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:28.405553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864258+0000","last_change":"2024-09-11T00:20:24.025555+0000","last_active":"2024-09-11T00:22:28.864258+0000","last_peered":"2024-09-11T00:22:28.864258+0000","last_clean":"2024-09-11T00:22:28.864258+0000","last_became_active":"2024-09-11T00:20:24.025171+0000","last_became_peered":"2024-09-11T00:20:24.025171+0000","last_unstale":"2024-09-11T00:22:28.864258+0000","last_undegraded":"2024-09-11T00:22:28.864258+0000","last_fullsized":"2024-09-11T00:22:28.864258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:22.708985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864496+0000","last_change":"2024-09-11T00:20:37.442057+0000","last_active":"2024-09-11T00:22:28.864496+0000","last_peered":"2024-09-11T00:22:28.864496+0000","last_clean":"2024-09-11T00:22:28.864496+0000","last_became_active":"2024-09-11T00:20:37.441555+0000","last_became_peered":"2024-09-11T00:20:37.441555+0000","last_unstale":"2024-09-11T00:22:28.864496+0000","last_undegraded":"2024-09-11T00:22:28.864496+0000","last_fullsized":"2024-09-11T00:22:28.864496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:08.155763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611677+0000","last_change":"2024-09-11T00:20:27.071533+0000","last_active":"2024-09-11T00:22:56.611677+0000","last_peered":"2024-09-11T00:22:56.611677+0000","last_clean":"2024-09-11T00:22:56.611677+0000","last_became_active":"2024-09-11T00:20:27.068771+0000","last_became_peered":"2024-09-11T00:20:27.068771+0000","last_unstale":"2024-09-11T00:22:56.611677+0000","last_undegraded":"2024-09-11T00:22:56.611677+0000","last_fullsized":"2024-09-11T00:22:56.611677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:05.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679914+0000","last_change":"2024-09-11T00:20:24.023282+0000","last_active":"2024-09-11T00:22:28.679914+0000","last_peered":"2024-09-11T00:22:28.679914+0000","last_clean":"2024-09-11T00:22:28.679914+0000","last_became_active":"2024-09-11T00:20:24.022914+0000","last_became_peered":"2024-09-11T00:20:24.022914+0000","last_unstale":"2024-09-11T00:22:28.679914+0000","last_undegraded":"2024-09-11T00:22:28.679914+0000","last_fullsized":"2024-09-11T00:22:28.679914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:45.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864811+0000","last_change":"2024-09-11T00:20:37.457338+0000","last_active":"2024-09-11T00:22:28.864811+0000","last_peered":"2024-09-11T00:22:28.864811+0000","last_clean":"2024-09-11T00:22:28.864811+0000","last_became_active":"2024-09-11T00:20:37.456993+0000","last_became_peered":"2024-09-11T00:20:37.456993+0000","last_unstale":"2024-09-11T00:22:28.864811+0000","last_undegraded":"2024-09-11T00:22:28.864811+0000","last_fullsized":"2024-09-11T00:22:28.864811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:57.284412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606431+0000","last_change":"2024-09-11T00:20:27.062752+0000","last_active":"2024-09-11T00:22:56.606431+0000","last_peered":"2024-09-11T00:22:56.606431+0000","last_clean":"2024-09-11T00:22:56.606431+0000","last_became_active":"2024-09-11T00:20:27.061569+0000","last_became_peered":"2024-09-11T00:20:27.061569+0000","last_unstale":"2024-09-11T00:22:56.606431+0000","last_undegraded":"2024-09-11T00:22:56.606431+0000","last_fullsized":"2024-09-11T00:22:56.606431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:30.904423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864019+0000","last_change":"2024-09-11T00:20:24.023197+0000","last_active":"2024-09-11T00:22:28.864019+0000","last_peered":"2024-09-11T00:22:28.864019+0000","last_clean":"2024-09-11T00:22:28.864019+0000","last_became_active":"2024-09-11T00:20:24.022867+0000","last_became_peered":"2024-09-11T00:20:24.022867+0000","last_unstale":"2024-09-11T00:22:28.864019+0000","last_undegraded":"2024-09-11T00:22:28.864019+0000","last_fullsized":"2024-09-11T00:22:28.864019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:37.949437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680695+0000","last_change":"2024-09-11T00:20:37.451731+0000","last_active":"2024-09-11T00:22:28.680695+0000","last_peered":"2024-09-11T00:22:28.680695+0000","last_clean":"2024-09-11T00:22:28.680695+0000","last_became_active":"2024-09-11T00:20:37.447710+0000","last_became_peered":"2024-09-11T00:20:37.447710+0000","last_unstale":"2024-09-11T00:22:28.680695+0000","last_undegraded":"2024-09-11T00:22:28.680695+0000","last_fullsized":"2024-09-11T00:22:28.680695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:11:38.471216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.565404+0000","last_change":"2024-09-11T00:20:27.066021+0000","last_active":"2024-09-11T00:22:56.565404+0000","last_peered":"2024-09-11T00:22:56.565404+0000","last_clean":"2024-09-11T00:22:56.565404+0000","last_became_active":"2024-09-11T00:20:27.065652+0000","last_became_peered":"2024-09-11T00:20:27.065652+0000","last_unstale":"2024-09-11T00:22:56.565404+0000","last_undegraded":"2024-09-11T00:22:56.565404+0000","last_fullsized":"2024-09-11T00:22:56.565404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:01.933031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864139+0000","last_change":"2024-09-11T00:20:24.035619+0000","last_active":"2024-09-11T00:22:28.864139+0000","last_peered":"2024-09-11T00:22:28.864139+0000","last_clean":"2024-09-11T00:22:28.864139+0000","last_became_active":"2024-09-11T00:20:24.035358+0000","last_became_peered":"2024-09-11T00:20:24.035358+0000","last_unstale":"2024-09-11T00:22:28.864139+0000","last_undegraded":"2024-09-11T00:22:28.864139+0000","last_fullsized":"2024-09-11T00:22:28.864139+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:56.890058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865646+0000","last_change":"2024-09-11T00:21:28.226415+0000","last_active":"2024-09-11T00:22:28.865646+0000","last_peered":"2024-09-11T00:22:28.865646+0000","last_clean":"2024-09-11T00:22:28.865646+0000","last_became_active":"2024-09-11T00:21:28.225894+0000","last_became_peered":"2024-09-11T00:21:28.225894+0000","last_unstale":"2024-09-11T00:22:28.865646+0000","last_undegraded":"2024-09-11T00:22:28.865646+0000","last_fullsized":"2024-09-11T00:22:28.865646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:59.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606794+0000","last_change":"2024-09-11T00:22:28.866891+0000","last_active":"2024-09-11T00:22:56.606794+0000","last_peered":"2024-09-11T00:22:56.606794+0000","last_clean":"2024-09-11T00:22:56.606794+0000","last_became_active":"2024-09-11T00:22:28.866541+0000","last_became_peered":"2024-09-11T00:22:28.866541+0000","last_unstale":"2024-09-11T00:22:56.606794+0000","last_undegraded":"2024-09-11T00:22:56.606794+0000","last_fullsized":"2024-09-11T00:22:56.606794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:35.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865567+0000","last_change":"2024-09-11T00:20:24.535701+0000","last_active":"2024-09-11T00:22:28.865567+0000","last_peered":"2024-09-11T00:22:28.865567+0000","last_clean":"2024-09-11T00:22:28.865567+0000","last_became_active":"2024-09-11T00:20:24.535389+0000","last_became_peered":"2024-09-11T00:20:24.535389+0000","last_unstale":"2024-09-11T00:22:28.865567+0000","last_undegraded":"2024-09-11T00:22:28.865567+0000","last_fullsized":"2024-09-11T00:22:28.865567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:29.044747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679639+0000","last_change":"2024-09-11T00:20:37.435584+0000","last_active":"2024-09-11T00:22:28.679639+0000","last_peered":"2024-09-11T00:22:28.679639+0000","last_clean":"2024-09-11T00:22:28.679639+0000","last_became_active":"2024-09-11T00:20:37.435237+0000","last_became_peered":"2024-09-11T00:20:37.435237+0000","last_unstale":"2024-09-11T00:22:28.679639+0000","last_undegraded":"2024-09-11T00:22:28.679639+0000","last_fullsized":"2024-09-11T00:22:28.679639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:34.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610308+0000","last_change":"2024-09-11T00:20:27.068718+0000","last_active":"2024-09-11T00:22:56.610308+0000","last_peered":"2024-09-11T00:22:56.610308+0000","last_clean":"2024-09-11T00:22:56.610308+0000","last_became_active":"2024-09-11T00:20:27.058921+0000","last_became_peered":"2024-09-11T00:20:27.058921+0000","last_unstale":"2024-09-11T00:22:56.610308+0000","last_undegraded":"2024-09-11T00:22:56.610308+0000","last_fullsized":"2024-09-11T00:22:56.610308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:16.056742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:57.080412+0000","last_change":"2024-09-11T00:20:24.023580+0000","last_active":"2024-09-11T00:22:57.080412+0000","last_peered":"2024-09-11T00:22:57.080412+0000","last_clean":"2024-09-11T00:22:57.080412+0000","last_became_active":"2024-09-11T00:20:24.023029+0000","last_became_peered":"2024-09-11T00:20:24.023029+0000","last_unstale":"2024-09-11T00:22:57.080412+0000","last_undegraded":"2024-09-11T00:22:57.080412+0000","last_fullsized":"2024-09-11T00:22:57.080412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:05.574228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863468+0000","last_change":"2024-09-11T00:20:37.449229+0000","last_active":"2024-09-11T00:22:28.863468+0000","last_peered":"2024-09-11T00:22:28.863468+0000","last_clean":"2024-09-11T00:22:28.863468+0000","last_became_active":"2024-09-11T00:20:37.447464+0000","last_became_peered":"2024-09-11T00:20:37.447464+0000","last_unstale":"2024-09-11T00:22:28.863468+0000","last_undegraded":"2024-09-11T00:22:28.863468+0000","last_fullsized":"2024-09-11T00:22:28.863468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:47.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597838+0000","last_change":"2024-09-11T00:20:27.063518+0000","last_active":"2024-09-11T00:22:56.597838+0000","last_peered":"2024-09-11T00:22:56.597838+0000","last_clean":"2024-09-11T00:22:56.597838+0000","last_became_active":"2024-09-11T00:20:27.063122+0000","last_became_peered":"2024-09-11T00:20:27.063122+0000","last_unstale":"2024-09-11T00:22:56.597838+0000","last_undegraded":"2024-09-11T00:22:56.597838+0000","last_fullsized":"2024-09-11T00:22:56.597838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:52.269862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864785+0000","last_change":"2024-09-11T00:20:24.035835+0000","last_active":"2024-09-11T00:22:28.864785+0000","last_peered":"2024-09-11T00:22:28.864785+0000","last_clean":"2024-09-11T00:22:28.864785+0000","last_became_active":"2024-09-11T00:20:24.035471+0000","last_became_peered":"2024-09-11T00:20:24.035471+0000","last_unstale":"2024-09-11T00:22:28.864785+0000","last_undegraded":"2024-09-11T00:22:28.864785+0000","last_fullsized":"2024-09-11T00:22:28.864785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:37.114863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863908+0000","last_change":"2024-09-11T00:20:37.439609+0000","last_active":"2024-09-11T00:22:28.863908+0000","last_peered":"2024-09-11T00:22:28.863908+0000","last_clean":"2024-09-11T00:22:28.863908+0000","last_became_active":"2024-09-11T00:20:37.439139+0000","last_became_peered":"2024-09-11T00:20:37.439139+0000","last_unstale":"2024-09-11T00:22:28.863908+0000","last_undegraded":"2024-09-11T00:22:28.863908+0000","last_fullsized":"2024-09-11T00:22:28.863908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:19.152096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610140+0000","last_change":"2024-09-11T00:20:27.062879+0000","last_active":"2024-09-11T00:22:56.610140+0000","last_peered":"2024-09-11T00:22:56.610140+0000","last_clean":"2024-09-11T00:22:56.610140+0000","last_became_active":"2024-09-11T00:20:27.062200+0000","last_became_peered":"2024-09-11T00:20:27.062200+0000","last_unstale":"2024-09-11T00:22:56.610140+0000","last_undegraded":"2024-09-11T00:22:56.610140+0000","last_fullsized":"2024-09-11T00:22:56.610140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:57.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680751+0000","last_change":"2024-09-11T00:21:28.230802+0000","last_active":"2024-09-11T00:22:28.680751+0000","last_peered":"2024-09-11T00:22:28.680751+0000","last_clean":"2024-09-11T00:22:28.680751+0000","last_became_active":"2024-09-11T00:21:28.230378+0000","last_became_peered":"2024-09-11T00:21:28.230378+0000","last_unstale":"2024-09-11T00:22:28.680751+0000","last_undegraded":"2024-09-11T00:22:28.680751+0000","last_fullsized":"2024-09-11T00:22:28.680751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:35.676964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864428+0000","last_change":"2024-09-11T00:20:37.444912+0000","last_active":"2024-09-11T00:22:28.864428+0000","last_peered":"2024-09-11T00:22:28.864428+0000","last_clean":"2024-09-11T00:22:28.864428+0000","last_became_active":"2024-09-11T00:20:37.444472+0000","last_became_peered":"2024-09-11T00:20:37.444472+0000","last_unstale":"2024-09-11T00:22:28.864428+0000","last_undegraded":"2024-09-11T00:22:28.864428+0000","last_fullsized":"2024-09-11T00:22:28.864428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:23.263981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606705+0000","last_change":"2024-09-11T00:20:27.060477+0000","last_active":"2024-09-11T00:22:56.606705+0000","last_peered":"2024-09-11T00:22:56.606705+0000","last_clean":"2024-09-11T00:22:56.606705+0000","last_became_active":"2024-09-11T00:20:27.060128+0000","last_became_peered":"2024-09-11T00:20:27.060128+0000","last_unstale":"2024-09-11T00:22:56.606705+0000","last_undegraded":"2024-09-11T00:22:56.606705+0000","last_fullsized":"2024-09-11T00:22:56.606705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:55.967799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866156+0000","last_change":"2024-09-11T00:20:24.028229+0000","last_active":"2024-09-11T00:22:28.866156+0000","last_peered":"2024-09-11T00:22:28.866156+0000","last_clean":"2024-09-11T00:22:28.866156+0000","last_became_active":"2024-09-11T00:20:24.027832+0000","last_became_peered":"2024-09-11T00:20:24.027832+0000","last_unstale":"2024-09-11T00:22:28.866156+0000","last_undegraded":"2024-09-11T00:22:28.866156+0000","last_fullsized":"2024-09-11T00:22:28.866156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:19.960403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682873+0000","last_change":"2024-09-11T00:20:37.442414+0000","last_active":"2024-09-11T00:22:28.682873+0000","last_peered":"2024-09-11T00:22:28.682873+0000","last_clean":"2024-09-11T00:22:28.682873+0000","last_became_active":"2024-09-11T00:20:37.442021+0000","last_became_peered":"2024-09-11T00:20:37.442021+0000","last_unstale":"2024-09-11T00:22:28.682873+0000","last_undegraded":"2024-09-11T00:22:28.682873+0000","last_fullsized":"2024-09-11T00:22:28.682873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:21.296638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.592369+0000","last_change":"2024-09-11T00:21:28.233303+0000","last_active":"2024-09-11T00:22:56.592369+0000","last_peered":"2024-09-11T00:22:56.592369+0000","last_clean":"2024-09-11T00:22:56.592369+0000","last_became_active":"2024-09-11T00:21:28.232039+0000","last_became_peered":"2024-09-11T00:21:28.232039+0000","last_unstale":"2024-09-11T00:22:56.592369+0000","last_undegraded":"2024-09-11T00:22:56.592369+0000","last_fullsized":"2024-09-11T00:22:56.592369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:36.828102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:29.403808+0000","last_change":"2024-09-11T00:20:24.037218+0000","last_active":"2024-09-11T00:23:29.403808+0000","last_peered":"2024-09-11T00:23:29.403808+0000","last_clean":"2024-09-11T00:23:29.403808+0000","last_became_active":"2024-09-11T00:20:24.036005+0000","last_became_peered":"2024-09-11T00:20:24.036005+0000","last_unstale":"2024-09-11T00:23:29.403808+0000","last_undegraded":"2024-09-11T00:23:29.403808+0000","last_fullsized":"2024-09-11T00:23:29.403808+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:17.758812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683067+0000","last_change":"2024-09-11T00:20:37.445584+0000","last_active":"2024-09-11T00:22:28.683067+0000","last_peered":"2024-09-11T00:22:28.683067+0000","last_clean":"2024-09-11T00:22:28.683067+0000","last_became_active":"2024-09-11T00:20:37.445196+0000","last_became_peered":"2024-09-11T00:20:37.445196+0000","last_unstale":"2024-09-11T00:22:28.683067+0000","last_undegraded":"2024-09-11T00:22:28.683067+0000","last_fullsized":"2024-09-11T00:22:28.683067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:05.966282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.598168+0000","last_change":"2024-09-11T00:20:27.060428+0000","last_active":"2024-09-11T00:22:56.598168+0000","last_peered":"2024-09-11T00:22:56.598168+0000","last_clean":"2024-09-11T00:22:56.598168+0000","last_became_active":"2024-09-11T00:20:27.060057+0000","last_became_peered":"2024-09-11T00:20:27.060057+0000","last_unstale":"2024-09-11T00:22:56.598168+0000","last_undegraded":"2024-09-11T00:22:56.598168+0000","last_fullsized":"2024-09-11T00:22:56.598168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:37.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865091+0000","last_change":"2024-09-11T00:20:24.025496+0000","last_active":"2024-09-11T00:22:28.865091+0000","last_peered":"2024-09-11T00:22:28.865091+0000","last_clean":"2024-09-11T00:22:28.865091+0000","last_became_active":"2024-09-11T00:20:24.025158+0000","last_became_peered":"2024-09-11T00:20:24.025158+0000","last_unstale":"2024-09-11T00:22:28.865091+0000","last_undegraded":"2024-09-11T00:22:28.865091+0000","last_fullsized":"2024-09-11T00:22:28.865091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:40.439708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734460+0000","last_change":"2024-09-11T00:20:37.453118+0000","last_active":"2024-09-11T00:22:28.734460+0000","last_peered":"2024-09-11T00:22:28.734460+0000","last_clean":"2024-09-11T00:22:28.734460+0000","last_became_active":"2024-09-11T00:20:37.452669+0000","last_became_peered":"2024-09-11T00:20:37.452669+0000","last_unstale":"2024-09-11T00:22:28.734460+0000","last_undegraded":"2024-09-11T00:22:28.734460+0000","last_fullsized":"2024-09-11T00:22:28.734460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:51:58.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606977+0000","last_change":"2024-09-11T00:20:27.069708+0000","last_active":"2024-09-11T00:22:56.606977+0000","last_peered":"2024-09-11T00:22:56.606977+0000","last_clean":"2024-09-11T00:22:56.606977+0000","last_became_active":"2024-09-11T00:20:27.067500+0000","last_became_peered":"2024-09-11T00:20:27.067500+0000","last_unstale":"2024-09-11T00:22:56.606977+0000","last_undegraded":"2024-09-11T00:22:56.606977+0000","last_fullsized":"2024-09-11T00:22:56.606977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:57.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864981+0000","last_change":"2024-09-11T00:20:24.023504+0000","last_active":"2024-09-11T00:22:28.864981+0000","last_peered":"2024-09-11T00:22:28.864981+0000","last_clean":"2024-09-11T00:22:28.864981+0000","last_became_active":"2024-09-11T00:20:24.022706+0000","last_became_peered":"2024-09-11T00:20:24.022706+0000","last_unstale":"2024-09-11T00:22:28.864981+0000","last_undegraded":"2024-09-11T00:22:28.864981+0000","last_fullsized":"2024-09-11T00:22:28.864981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:21.216325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865351+0000","last_change":"2024-09-11T00:19:05.531191+0000","last_active":"2024-09-11T00:22:28.865351+0000","last_peered":"2024-09-11T00:22:28.865351+0000","last_clean":"2024-09-11T00:22:28.865351+0000","last_became_active":"2024-09-11T00:19:05.525664+0000","last_became_peered":"2024-09-11T00:19:05.525664+0000","last_unstale":"2024-09-11T00:22:28.865351+0000","last_undegraded":"2024-09-11T00:22:28.865351+0000","last_fullsized":"2024-09-11T00:22:28.865351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_clean_scrub_stamp":"2024-09-11T00:15:23.365345+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:06.253537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866037+0000","last_change":"2024-09-11T00:20:37.446403+0000","last_active":"2024-09-11T00:22:28.866037+0000","last_peered":"2024-09-11T00:22:28.866037+0000","last_clean":"2024-09-11T00:22:28.866037+0000","last_became_active":"2024-09-11T00:20:37.446029+0000","last_became_peered":"2024-09-11T00:20:37.446029+0000","last_unstale":"2024-09-11T00:22:28.866037+0000","last_undegraded":"2024-09-11T00:22:28.866037+0000","last_fullsized":"2024-09-11T00:22:28.866037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611496+0000","last_change":"2024-09-11T00:20:27.063086+0000","last_active":"2024-09-11T00:22:56.611496+0000","last_peered":"2024-09-11T00:22:56.611496+0000","last_clean":"2024-09-11T00:22:56.611496+0000","last_became_active":"2024-09-11T00:20:27.062790+0000","last_became_peered":"2024-09-11T00:20:27.062790+0000","last_unstale":"2024-09-11T00:22:56.611496+0000","last_undegraded":"2024-09-11T00:22:56.611496+0000","last_fullsized":"2024-09-11T00:22:56.611496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:36.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679645+0000","last_change":"2024-09-11T00:20:24.535837+0000","last_active":"2024-09-11T00:22:28.679645+0000","last_peered":"2024-09-11T00:22:28.679645+0000","last_clean":"2024-09-11T00:22:28.679645+0000","last_became_active":"2024-09-11T00:20:24.535308+0000","last_became_peered":"2024-09-11T00:20:24.535308+0000","last_unstale":"2024-09-11T00:22:28.679645+0000","last_undegraded":"2024-09-11T00:22:28.679645+0000","last_fullsized":"2024-09-11T00:22:28.679645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.807570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864240+0000","last_change":"2024-09-11T00:20:37.443844+0000","last_active":"2024-09-11T00:22:28.864240+0000","last_peered":"2024-09-11T00:22:28.864240+0000","last_clean":"2024-09-11T00:22:28.864240+0000","last_became_active":"2024-09-11T00:20:37.443374+0000","last_became_peered":"2024-09-11T00:20:37.443374+0000","last_unstale":"2024-09-11T00:22:28.864240+0000","last_undegraded":"2024-09-11T00:22:28.864240+0000","last_fullsized":"2024-09-11T00:22:28.864240+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:47.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610053+0000","last_change":"2024-09-11T00:20:27.072874+0000","last_active":"2024-09-11T00:22:56.610053+0000","last_peered":"2024-09-11T00:22:56.610053+0000","last_clean":"2024-09-11T00:22:56.610053+0000","last_became_active":"2024-09-11T00:20:27.072454+0000","last_became_peered":"2024-09-11T00:20:27.072454+0000","last_unstale":"2024-09-11T00:22:56.610053+0000","last_undegraded":"2024-09-11T00:22:56.610053+0000","last_fullsized":"2024-09-11T00:22:56.610053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:25.731037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734678+0000","last_change":"2024-09-11T00:20:24.018929+0000","last_active":"2024-09-11T00:22:28.734678+0000","last_peered":"2024-09-11T00:22:28.734678+0000","last_clean":"2024-09-11T00:22:28.734678+0000","last_became_active":"2024-09-11T00:20:24.018575+0000","last_became_peered":"2024-09-11T00:20:24.018575+0000","last_unstale":"2024-09-11T00:22:28.734678+0000","last_undegraded":"2024-09-11T00:22:28.734678+0000","last_fullsized":"2024-09-11T00:22:28.734678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.695561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679083+0000","last_change":"2024-09-11T00:20:37.453468+0000","last_active":"2024-09-11T00:22:28.679083+0000","last_peered":"2024-09-11T00:22:28.679083+0000","last_clean":"2024-09-11T00:22:28.679083+0000","last_became_active":"2024-09-11T00:20:37.453007+0000","last_became_peered":"2024-09-11T00:20:37.453007+0000","last_unstale":"2024-09-11T00:22:28.679083+0000","last_undegraded":"2024-09-11T00:22:28.679083+0000","last_fullsized":"2024-09-11T00:22:28.679083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:22.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604628+0000","last_change":"2024-09-11T00:20:27.064618+0000","last_active":"2024-09-11T00:22:56.604628+0000","last_peered":"2024-09-11T00:22:56.604628+0000","last_clean":"2024-09-11T00:22:56.604628+0000","last_became_active":"2024-09-11T00:20:27.064310+0000","last_became_peered":"2024-09-11T00:20:27.064310+0000","last_unstale":"2024-09-11T00:22:56.604628+0000","last_undegraded":"2024-09-11T00:22:56.604628+0000","last_fullsized":"2024-09-11T00:22:56.604628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:41.320561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683037+0000","last_change":"2024-09-11T00:20:24.023522+0000","last_active":"2024-09-11T00:22:28.683037+0000","last_peered":"2024-09-11T00:22:28.683037+0000","last_clean":"2024-09-11T00:22:28.683037+0000","last_became_active":"2024-09-11T00:20:24.022694+0000","last_became_peered":"2024-09-11T00:20:24.022694+0000","last_unstale":"2024-09-11T00:22:28.683037+0000","last_undegraded":"2024-09-11T00:22:28.683037+0000","last_fullsized":"2024-09-11T00:22:28.683037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:12.374179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733919+0000","last_change":"2024-09-11T00:20:37.441239+0000","last_active":"2024-09-11T00:22:28.733919+0000","last_peered":"2024-09-11T00:22:28.733919+0000","last_clean":"2024-09-11T00:22:28.733919+0000","last_became_active":"2024-09-11T00:20:37.440825+0000","last_became_peered":"2024-09-11T00:20:37.440825+0000","last_unstale":"2024-09-11T00:22:28.733919+0000","last_undegraded":"2024-09-11T00:22:28.733919+0000","last_fullsized":"2024-09-11T00:22:28.733919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:04.972569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594629+0000","last_change":"2024-09-11T00:20:27.059780+0000","last_active":"2024-09-11T00:22:56.594629+0000","last_peered":"2024-09-11T00:22:56.594629+0000","last_clean":"2024-09-11T00:22:56.594629+0000","last_became_active":"2024-09-11T00:20:27.059411+0000","last_became_peered":"2024-09-11T00:20:27.059411+0000","last_unstale":"2024-09-11T00:22:56.594629+0000","last_undegraded":"2024-09-11T00:22:56.594629+0000","last_fullsized":"2024-09-11T00:22:56.594629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:42.058883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864514+0000","last_change":"2024-09-11T00:20:24.024115+0000","last_active":"2024-09-11T00:22:28.864514+0000","last_peered":"2024-09-11T00:22:28.864514+0000","last_clean":"2024-09-11T00:22:28.864514+0000","last_became_active":"2024-09-11T00:20:24.023729+0000","last_became_peered":"2024-09-11T00:20:24.023729+0000","last_unstale":"2024-09-11T00:22:28.864514+0000","last_undegraded":"2024-09-11T00:22:28.864514+0000","last_fullsized":"2024-09-11T00:22:28.864514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:15.874168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733904+0000","last_change":"2024-09-11T00:20:37.443591+0000","last_active":"2024-09-11T00:22:28.733904+0000","last_peered":"2024-09-11T00:22:28.733904+0000","last_clean":"2024-09-11T00:22:28.733904+0000","last_became_active":"2024-09-11T00:20:37.443115+0000","last_became_peered":"2024-09-11T00:20:37.443115+0000","last_unstale":"2024-09-11T00:22:28.733904+0000","last_undegraded":"2024-09-11T00:22:28.733904+0000","last_fullsized":"2024-09-11T00:22:28.733904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:34.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607526+0000","last_change":"2024-09-11T00:20:27.061676+0000","last_active":"2024-09-11T00:22:56.607526+0000","last_peered":"2024-09-11T00:22:56.607526+0000","last_clean":"2024-09-11T00:22:56.607526+0000","last_became_active":"2024-09-11T00:20:27.061372+0000","last_became_peered":"2024-09-11T00:20:27.061372+0000","last_unstale":"2024-09-11T00:22:56.607526+0000","last_undegraded":"2024-09-11T00:22:56.607526+0000","last_fullsized":"2024-09-11T00:22:56.607526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:24.296673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863776+0000","last_change":"2024-09-11T00:21:28.244227+0000","last_active":"2024-09-11T00:22:28.863776+0000","last_peered":"2024-09-11T00:22:28.863776+0000","last_clean":"2024-09-11T00:22:28.863776+0000","last_became_active":"2024-09-11T00:21:28.231547+0000","last_became_peered":"2024-09-11T00:21:28.231547+0000","last_unstale":"2024-09-11T00:22:28.863776+0000","last_undegraded":"2024-09-11T00:22:28.863776+0000","last_fullsized":"2024-09-11T00:22:28.863776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:35.119308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682992+0000","last_change":"2024-09-11T00:20:37.454603+0000","last_active":"2024-09-11T00:22:28.682992+0000","last_peered":"2024-09-11T00:22:28.682992+0000","last_clean":"2024-09-11T00:22:28.682992+0000","last_became_active":"2024-09-11T00:20:37.454123+0000","last_became_peered":"2024-09-11T00:20:37.454123+0000","last_unstale":"2024-09-11T00:22:28.682992+0000","last_undegraded":"2024-09-11T00:22:28.682992+0000","last_fullsized":"2024-09-11T00:22:28.682992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:09.111768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594320+0000","last_change":"2024-09-11T00:20:27.058806+0000","last_active":"2024-09-11T00:22:56.594320+0000","last_peered":"2024-09-11T00:22:56.594320+0000","last_clean":"2024-09-11T00:22:56.594320+0000","last_became_active":"2024-09-11T00:20:27.058458+0000","last_became_peered":"2024-09-11T00:20:27.058458+0000","last_unstale":"2024-09-11T00:22:56.594320+0000","last_undegraded":"2024-09-11T00:22:56.594320+0000","last_fullsized":"2024-09-11T00:22:56.594320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:48.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":177,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:28.401683+0000","last_change":"2024-09-11T00:20:24.022328+0000","last_active":"2024-09-11T00:23:28.401683+0000","last_peered":"2024-09-11T00:23:28.401683+0000","last_clean":"2024-09-11T00:23:28.401683+0000","last_became_active":"2024-09-11T00:20:24.021908+0000","last_became_peered":"2024-09-11T00:20:24.021908+0000","last_unstale":"2024-09-11T00:23:28.401683+0000","last_undegraded":"2024-09-11T00:23:28.401683+0000","last_fullsized":"2024-09-11T00:23:28.401683+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:35.688767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680798+0000","last_change":"2024-09-11T00:20:37.444898+0000","last_active":"2024-09-11T00:22:28.680798+0000","last_peered":"2024-09-11T00:22:28.680798+0000","last_clean":"2024-09-11T00:22:28.680798+0000","last_became_active":"2024-09-11T00:20:37.443840+0000","last_became_peered":"2024-09-11T00:20:37.443840+0000","last_unstale":"2024-09-11T00:22:28.680798+0000","last_undegraded":"2024-09-11T00:22:28.680798+0000","last_fullsized":"2024-09-11T00:22:28.680798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:58.668882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606050+0000","last_change":"2024-09-11T00:20:27.061198+0000","last_active":"2024-09-11T00:22:56.606050+0000","last_peered":"2024-09-11T00:22:56.606050+0000","last_clean":"2024-09-11T00:22:56.606050+0000","last_became_active":"2024-09-11T00:20:27.060877+0000","last_became_peered":"2024-09-11T00:20:27.060877+0000","last_unstale":"2024-09-11T00:22:56.606050+0000","last_undegraded":"2024-09-11T00:22:56.606050+0000","last_fullsized":"2024-09-11T00:22:56.606050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:03.595262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734264+0000","last_change":"2024-09-11T00:20:24.022755+0000","last_active":"2024-09-11T00:22:28.734264+0000","last_peered":"2024-09-11T00:22:28.734264+0000","last_clean":"2024-09-11T00:22:28.734264+0000","last_became_active":"2024-09-11T00:20:24.021530+0000","last_became_peered":"2024-09-11T00:20:24.021530+0000","last_unstale":"2024-09-11T00:22:28.734264+0000","last_undegraded":"2024-09-11T00:22:28.734264+0000","last_fullsized":"2024-09-11T00:22:28.734264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:33.881000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864371+0000","last_change":"2024-09-11T00:20:37.443722+0000","last_active":"2024-09-11T00:22:28.864371+0000","last_peered":"2024-09-11T00:22:28.864371+0000","last_clean":"2024-09-11T00:22:28.864371+0000","last_became_active":"2024-09-11T00:20:37.443339+0000","last_became_peered":"2024-09-11T00:20:37.443339+0000","last_unstale":"2024-09-11T00:22:28.864371+0000","last_undegraded":"2024-09-11T00:22:28.864371+0000","last_fullsized":"2024-09-11T00:22:28.864371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:30.221804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610517+0000","last_change":"2024-09-11T00:20:27.073158+0000","last_active":"2024-09-11T00:22:56.610517+0000","last_peered":"2024-09-11T00:22:56.610517+0000","last_clean":"2024-09-11T00:22:56.610517+0000","last_became_active":"2024-09-11T00:20:27.072805+0000","last_became_peered":"2024-09-11T00:20:27.072805+0000","last_unstale":"2024-09-11T00:22:56.610517+0000","last_undegraded":"2024-09-11T00:22:56.610517+0000","last_fullsized":"2024-09-11T00:22:56.610517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:35.124632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679703+0000","last_change":"2024-09-11T00:20:24.027178+0000","last_active":"2024-09-11T00:22:28.679703+0000","last_peered":"2024-09-11T00:22:28.679703+0000","last_clean":"2024-09-11T00:22:28.679703+0000","last_became_active":"2024-09-11T00:20:24.026821+0000","last_became_peered":"2024-09-11T00:20:24.026821+0000","last_unstale":"2024-09-11T00:22:28.679703+0000","last_undegraded":"2024-09-11T00:22:28.679703+0000","last_fullsized":"2024-09-11T00:22:28.679703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:38.393135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734422+0000","last_change":"2024-09-11T00:20:37.447587+0000","last_active":"2024-09-11T00:22:28.734422+0000","last_peered":"2024-09-11T00:22:28.734422+0000","last_clean":"2024-09-11T00:22:28.734422+0000","last_became_active":"2024-09-11T00:20:37.447166+0000","last_became_peered":"2024-09-11T00:20:37.447166+0000","last_unstale":"2024-09-11T00:22:28.734422+0000","last_undegraded":"2024-09-11T00:22:28.734422+0000","last_fullsized":"2024-09-11T00:22:28.734422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:47.469539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605796+0000","last_change":"2024-09-11T00:20:27.059914+0000","last_active":"2024-09-11T00:22:56.605796+0000","last_peered":"2024-09-11T00:22:56.605796+0000","last_clean":"2024-09-11T00:22:56.605796+0000","last_became_active":"2024-09-11T00:20:27.059595+0000","last_became_peered":"2024-09-11T00:20:27.059595+0000","last_unstale":"2024-09-11T00:22:56.605796+0000","last_undegraded":"2024-09-11T00:22:56.605796+0000","last_fullsized":"2024-09-11T00:22:56.605796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.360010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'38","reported_seq":343,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:28.424758+0000","last_change":"2024-09-11T00:20:24.023094+0000","last_active":"2024-09-11T00:23:28.424758+0000","last_peered":"2024-09-11T00:23:28.424758+0000","last_clean":"2024-09-11T00:23:28.424758+0000","last_became_active":"2024-09-11T00:20:24.022742+0000","last_became_peered":"2024-09-11T00:20:24.022742+0000","last_unstale":"2024-09-11T00:23:28.424758+0000","last_undegraded":"2024-09-11T00:23:28.424758+0000","last_fullsized":"2024-09-11T00:23:28.424758+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:41.354435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62632,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":259,"num_read_kb":191,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680599+0000","last_change":"2024-09-11T00:20:37.451610+0000","last_active":"2024-09-11T00:22:28.680599+0000","last_peered":"2024-09-11T00:22:28.680599+0000","last_clean":"2024-09-11T00:22:28.680599+0000","last_became_active":"2024-09-11T00:20:37.444233+0000","last_became_peered":"2024-09-11T00:20:37.444233+0000","last_unstale":"2024-09-11T00:22:28.680599+0000","last_undegraded":"2024-09-11T00:22:28.680599+0000","last_fullsized":"2024-09-11T00:22:28.680599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:02.558457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608959+0000","last_change":"2024-09-11T00:20:27.061229+0000","last_active":"2024-09-11T00:22:56.608959+0000","last_peered":"2024-09-11T00:22:56.608959+0000","last_clean":"2024-09-11T00:22:56.608959+0000","last_became_active":"2024-09-11T00:20:27.060360+0000","last_became_peered":"2024-09-11T00:20:27.060360+0000","last_unstale":"2024-09-11T00:22:56.608959+0000","last_undegraded":"2024-09-11T00:22:56.608959+0000","last_fullsized":"2024-09-11T00:22:56.608959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.107405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734591+0000","last_change":"2024-09-11T00:20:24.036483+0000","last_active":"2024-09-11T00:22:28.734591+0000","last_peered":"2024-09-11T00:22:28.734591+0000","last_clean":"2024-09-11T00:22:28.734591+0000","last_became_active":"2024-09-11T00:20:24.035855+0000","last_became_peered":"2024-09-11T00:20:24.035855+0000","last_unstale":"2024-09-11T00:22:28.734591+0000","last_undegraded":"2024-09-11T00:22:28.734591+0000","last_fullsized":"2024-09-11T00:22:28.734591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.611169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864762+0000","last_change":"2024-09-11T00:21:28.245665+0000","last_active":"2024-09-11T00:22:28.864762+0000","last_peered":"2024-09-11T00:22:28.864762+0000","last_clean":"2024-09-11T00:22:28.864762+0000","last_became_active":"2024-09-11T00:21:28.245221+0000","last_became_peered":"2024-09-11T00:21:28.245221+0000","last_unstale":"2024-09-11T00:22:28.864762+0000","last_undegraded":"2024-09-11T00:22:28.864762+0000","last_fullsized":"2024-09-11T00:22:28.864762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:36.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611454+0000","last_change":"2024-09-11T00:21:28.231435+0000","last_active":"2024-09-11T00:22:56.611454+0000","last_peered":"2024-09-11T00:22:56.611454+0000","last_clean":"2024-09-11T00:22:56.611454+0000","last_became_active":"2024-09-11T00:21:28.230956+0000","last_became_peered":"2024-09-11T00:21:28.230956+0000","last_unstale":"2024-09-11T00:22:56.611454+0000","last_undegraded":"2024-09-11T00:22:56.611454+0000","last_fullsized":"2024-09-11T00:22:56.611454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:28.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865542+0000","last_change":"2024-09-11T00:20:24.023068+0000","last_active":"2024-09-11T00:22:28.865542+0000","last_peered":"2024-09-11T00:22:28.865542+0000","last_clean":"2024-09-11T00:22:28.865542+0000","last_became_active":"2024-09-11T00:20:24.022718+0000","last_became_peered":"2024-09-11T00:20:24.022718+0000","last_unstale":"2024-09-11T00:22:28.865542+0000","last_undegraded":"2024-09-11T00:22:28.865542+0000","last_fullsized":"2024-09-11T00:22:28.865542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.651536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863633+0000","last_change":"2024-09-11T00:21:28.231887+0000","last_active":"2024-09-11T00:22:28.863633+0000","last_peered":"2024-09-11T00:22:28.863633+0000","last_clean":"2024-09-11T00:22:28.863633+0000","last_became_active":"2024-09-11T00:21:28.231456+0000","last_became_peered":"2024-09-11T00:21:28.231456+0000","last_unstale":"2024-09-11T00:22:28.863633+0000","last_undegraded":"2024-09-11T00:22:28.863633+0000","last_fullsized":"2024-09-11T00:22:28.863633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:32.712598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613303+0000","last_change":"2024-09-11T00:20:27.065109+0000","last_active":"2024-09-11T00:22:56.613303+0000","last_peered":"2024-09-11T00:22:56.613303+0000","last_clean":"2024-09-11T00:22:56.613303+0000","last_became_active":"2024-09-11T00:20:27.064677+0000","last_became_peered":"2024-09-11T00:20:27.064677+0000","last_unstale":"2024-09-11T00:22:56.613303+0000","last_undegraded":"2024-09-11T00:22:56.613303+0000","last_fullsized":"2024-09-11T00:22:56.613303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:19.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":174,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:27.399849+0000","last_change":"2024-09-11T00:20:24.025347+0000","last_active":"2024-09-11T00:23:27.399849+0000","last_peered":"2024-09-11T00:23:27.399849+0000","last_clean":"2024-09-11T00:23:27.399849+0000","last_became_active":"2024-09-11T00:20:24.024980+0000","last_became_peered":"2024-09-11T00:20:24.024980+0000","last_unstale":"2024-09-11T00:23:27.399849+0000","last_undegraded":"2024-09-11T00:23:27.399849+0000","last_fullsized":"2024-09-11T00:23:27.399849+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:12.530339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863597+0000","last_change":"2024-09-11T00:20:37.440340+0000","last_active":"2024-09-11T00:22:28.863597+0000","last_peered":"2024-09-11T00:22:28.863597+0000","last_clean":"2024-09-11T00:22:28.863597+0000","last_became_active":"2024-09-11T00:20:37.439896+0000","last_became_peered":"2024-09-11T00:20:37.439896+0000","last_unstale":"2024-09-11T00:22:28.863597+0000","last_undegraded":"2024-09-11T00:22:28.863597+0000","last_fullsized":"2024-09-11T00:22:28.863597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:38.774065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679198+0000","last_change":"2024-09-11T00:20:37.445860+0000","last_active":"2024-09-11T00:22:28.679198+0000","last_peered":"2024-09-11T00:22:28.679198+0000","last_clean":"2024-09-11T00:22:28.679198+0000","last_became_active":"2024-09-11T00:20:37.445450+0000","last_became_peered":"2024-09-11T00:20:37.445450+0000","last_unstale":"2024-09-11T00:22:28.679198+0000","last_undegraded":"2024-09-11T00:22:28.679198+0000","last_fullsized":"2024-09-11T00:22:28.679198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:44.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607656+0000","last_change":"2024-09-11T00:20:27.073529+0000","last_active":"2024-09-11T00:22:56.607656+0000","last_peered":"2024-09-11T00:22:56.607656+0000","last_clean":"2024-09-11T00:22:56.607656+0000","last_became_active":"2024-09-11T00:20:27.073168+0000","last_became_peered":"2024-09-11T00:20:27.073168+0000","last_unstale":"2024-09-11T00:22:56.607656+0000","last_undegraded":"2024-09-11T00:22:56.607656+0000","last_fullsized":"2024-09-11T00:22:56.607656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:16.407543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734124+0000","last_change":"2024-09-11T00:20:24.021778+0000","last_active":"2024-09-11T00:22:28.734124+0000","last_peered":"2024-09-11T00:22:28.734124+0000","last_clean":"2024-09-11T00:22:28.734124+0000","last_became_active":"2024-09-11T00:20:24.021443+0000","last_became_peered":"2024-09-11T00:20:24.021443+0000","last_unstale":"2024-09-11T00:22:28.734124+0000","last_undegraded":"2024-09-11T00:22:28.734124+0000","last_fullsized":"2024-09-11T00:22:28.734124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864397+0000","last_change":"2024-09-11T00:20:37.444279+0000","last_active":"2024-09-11T00:22:28.864397+0000","last_peered":"2024-09-11T00:22:28.864397+0000","last_clean":"2024-09-11T00:22:28.864397+0000","last_became_active":"2024-09-11T00:20:37.443821+0000","last_became_peered":"2024-09-11T00:20:37.443821+0000","last_unstale":"2024-09-11T00:22:28.864397+0000","last_undegraded":"2024-09-11T00:22:28.864397+0000","last_fullsized":"2024-09-11T00:22:28.864397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:10.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606265+0000","last_change":"2024-09-11T00:20:27.066155+0000","last_active":"2024-09-11T00:22:56.606265+0000","last_peered":"2024-09-11T00:22:56.606265+0000","last_clean":"2024-09-11T00:22:56.606265+0000","last_became_active":"2024-09-11T00:20:27.057262+0000","last_became_peered":"2024-09-11T00:20:27.057262+0000","last_unstale":"2024-09-11T00:22:56.606265+0000","last_undegraded":"2024-09-11T00:22:56.606265+0000","last_fullsized":"2024-09-11T00:22:56.606265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:54.160215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682647+0000","last_change":"2024-09-11T00:20:24.026123+0000","last_active":"2024-09-11T00:22:28.682647+0000","last_peered":"2024-09-11T00:22:28.682647+0000","last_clean":"2024-09-11T00:22:28.682647+0000","last_became_active":"2024-09-11T00:20:24.025832+0000","last_became_peered":"2024-09-11T00:20:24.025832+0000","last_unstale":"2024-09-11T00:22:28.682647+0000","last_undegraded":"2024-09-11T00:22:28.682647+0000","last_fullsized":"2024-09-11T00:22:28.682647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:45.444840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830187+0000","last_change":"2024-09-11T00:20:24.028523+0000","last_active":"2024-09-11T00:22:28.830187+0000","last_peered":"2024-09-11T00:22:28.830187+0000","last_clean":"2024-09-11T00:22:28.830187+0000","last_became_active":"2024-09-11T00:20:24.027801+0000","last_became_peered":"2024-09-11T00:20:24.027801+0000","last_unstale":"2024-09-11T00:22:28.830187+0000","last_undegraded":"2024-09-11T00:22:28.830187+0000","last_fullsized":"2024-09-11T00:22:28.830187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:14.174983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607124+0000","last_change":"2024-09-11T00:20:27.074634+0000","last_active":"2024-09-11T00:22:56.607124+0000","last_peered":"2024-09-11T00:22:56.607124+0000","last_clean":"2024-09-11T00:22:56.607124+0000","last_became_active":"2024-09-11T00:20:27.074292+0000","last_became_peered":"2024-09-11T00:20:27.074292+0000","last_unstale":"2024-09-11T00:22:56.607124+0000","last_undegraded":"2024-09-11T00:22:56.607124+0000","last_fullsized":"2024-09-11T00:22:56.607124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:07.467262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679446+0000","last_change":"2024-09-11T00:20:37.454341+0000","last_active":"2024-09-11T00:22:28.679446+0000","last_peered":"2024-09-11T00:22:28.679446+0000","last_clean":"2024-09-11T00:22:28.679446+0000","last_became_active":"2024-09-11T00:20:37.453868+0000","last_became_peered":"2024-09-11T00:20:37.453868+0000","last_unstale":"2024-09-11T00:22:28.679446+0000","last_undegraded":"2024-09-11T00:22:28.679446+0000","last_fullsized":"2024-09-11T00:22:28.679446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.015338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":181,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:27.400519+0000","last_change":"2024-09-11T00:20:24.025184+0000","last_active":"2024-09-11T00:23:27.400519+0000","last_peered":"2024-09-11T00:23:27.400519+0000","last_clean":"2024-09-11T00:23:27.400519+0000","last_became_active":"2024-09-11T00:20:24.024670+0000","last_became_peered":"2024-09-11T00:20:24.024670+0000","last_unstale":"2024-09-11T00:23:27.400519+0000","last_undegraded":"2024-09-11T00:23:27.400519+0000","last_fullsized":"2024-09-11T00:23:27.400519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:40.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594092+0000","last_change":"2024-09-11T00:20:27.057822+0000","last_active":"2024-09-11T00:22:56.594092+0000","last_peered":"2024-09-11T00:22:56.594092+0000","last_clean":"2024-09-11T00:22:56.594092+0000","last_became_active":"2024-09-11T00:20:27.057518+0000","last_became_peered":"2024-09-11T00:20:27.057518+0000","last_unstale":"2024-09-11T00:22:56.594092+0000","last_undegraded":"2024-09-11T00:22:56.594092+0000","last_fullsized":"2024-09-11T00:22:56.594092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:00.264327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679539+0000","last_change":"2024-09-11T00:20:37.438369+0000","last_active":"2024-09-11T00:22:28.679539+0000","last_peered":"2024-09-11T00:22:28.679539+0000","last_clean":"2024-09-11T00:22:28.679539+0000","last_became_active":"2024-09-11T00:20:37.437942+0000","last_became_peered":"2024-09-11T00:20:37.437942+0000","last_unstale":"2024-09-11T00:22:28.679539+0000","last_undegraded":"2024-09-11T00:22:28.679539+0000","last_fullsized":"2024-09-11T00:22:28.679539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.410905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.595081+0000","last_change":"2024-09-11T00:20:27.057449+0000","last_active":"2024-09-11T00:22:56.595081+0000","last_peered":"2024-09-11T00:22:56.595081+0000","last_clean":"2024-09-11T00:22:56.595081+0000","last_became_active":"2024-09-11T00:20:27.057146+0000","last_became_peered":"2024-09-11T00:20:27.057146+0000","last_unstale":"2024-09-11T00:22:56.595081+0000","last_undegraded":"2024-09-11T00:22:56.595081+0000","last_fullsized":"2024-09-11T00:22:56.595081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:57.895062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.603116+0000","last_change":"2024-09-11T00:20:27.061515+0000","last_active":"2024-09-11T00:22:56.603116+0000","last_peered":"2024-09-11T00:22:56.603116+0000","last_clean":"2024-09-11T00:22:56.603116+0000","last_became_active":"2024-09-11T00:20:27.061175+0000","last_became_peered":"2024-09-11T00:20:27.061175+0000","last_unstale":"2024-09-11T00:22:56.603116+0000","last_undegraded":"2024-09-11T00:22:56.603116+0000","last_fullsized":"2024-09-11T00:22:56.603116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:03.112574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.609556+0000","last_change":"2024-09-11T00:20:27.063015+0000","last_active":"2024-09-11T00:22:56.609556+0000","last_peered":"2024-09-11T00:22:56.609556+0000","last_clean":"2024-09-11T00:22:56.609556+0000","last_became_active":"2024-09-11T00:20:27.062689+0000","last_became_peered":"2024-09-11T00:20:27.062689+0000","last_unstale":"2024-09-11T00:22:56.609556+0000","last_undegraded":"2024-09-11T00:22:56.609556+0000","last_fullsized":"2024-09-11T00:22:56.609556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.736073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605982+0000","last_change":"2024-09-11T00:20:27.062196+0000","last_active":"2024-09-11T00:22:56.605982+0000","last_peered":"2024-09-11T00:22:56.605982+0000","last_clean":"2024-09-11T00:22:56.605982+0000","last_became_active":"2024-09-11T00:20:27.061918+0000","last_became_peered":"2024-09-11T00:20:27.061918+0000","last_unstale":"2024-09-11T00:22:56.605982+0000","last_undegraded":"2024-09-11T00:22:56.605982+0000","last_fullsized":"2024-09-11T00:22:56.605982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:45.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611720+0000","last_change":"2024-09-11T00:20:27.059021+0000","last_active":"2024-09-11T00:22:56.611720+0000","last_peered":"2024-09-11T00:22:56.611720+0000","last_clean":"2024-09-11T00:22:56.611720+0000","last_became_active":"2024-09-11T00:20:27.058726+0000","last_became_peered":"2024-09-11T00:20:27.058726+0000","last_unstale":"2024-09-11T00:22:56.611720+0000","last_undegraded":"2024-09-11T00:22:56.611720+0000","last_fullsized":"2024-09-11T00:22:56.611720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:23.900236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.593828+0000","last_change":"2024-09-11T00:20:27.057593+0000","last_active":"2024-09-11T00:22:56.593828+0000","last_peered":"2024-09-11T00:22:56.593828+0000","last_clean":"2024-09-11T00:22:56.593828+0000","last_became_active":"2024-09-11T00:20:27.057310+0000","last_became_peered":"2024-09-11T00:20:27.057310+0000","last_unstale":"2024-09-11T00:22:56.593828+0000","last_undegraded":"2024-09-11T00:22:56.593828+0000","last_fullsized":"2024-09-11T00:22:56.593828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:52.355218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605460+0000","last_change":"2024-09-11T00:20:27.061140+0000","last_active":"2024-09-11T00:22:56.605460+0000","last_peered":"2024-09-11T00:22:56.605460+0000","last_clean":"2024-09-11T00:22:56.605460+0000","last_became_active":"2024-09-11T00:20:27.060500+0000","last_became_peered":"2024-09-11T00:20:27.060500+0000","last_unstale":"2024-09-11T00:22:56.605460+0000","last_undegraded":"2024-09-11T00:22:56.605460+0000","last_fullsized":"2024-09-11T00:22:56.605460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.159068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.591859+0000","last_change":"2024-09-11T00:20:27.061936+0000","last_active":"2024-09-11T00:22:56.591859+0000","last_peered":"2024-09-11T00:22:56.591859+0000","last_clean":"2024-09-11T00:22:56.591859+0000","last_became_active":"2024-09-11T00:20:27.061616+0000","last_became_peered":"2024-09-11T00:20:27.061616+0000","last_unstale":"2024-09-11T00:22:56.591859+0000","last_undegraded":"2024-09-11T00:22:56.591859+0000","last_fullsized":"2024-09-11T00:22:56.591859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:32.185634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608892+0000","last_change":"2024-09-11T00:20:27.074645+0000","last_active":"2024-09-11T00:22:56.608892+0000","last_peered":"2024-09-11T00:22:56.608892+0000","last_clean":"2024-09-11T00:22:56.608892+0000","last_became_active":"2024-09-11T00:20:27.074355+0000","last_became_peered":"2024-09-11T00:20:27.074355+0000","last_unstale":"2024-09-11T00:22:56.608892+0000","last_undegraded":"2024-09-11T00:22:56.608892+0000","last_fullsized":"2024-09-11T00:22:56.608892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.601812+0000","last_change":"2024-09-11T00:20:27.069588+0000","last_active":"2024-09-11T00:22:56.601812+0000","last_peered":"2024-09-11T00:22:56.601812+0000","last_clean":"2024-09-11T00:22:56.601812+0000","last_became_active":"2024-09-11T00:20:27.067403+0000","last_became_peered":"2024-09-11T00:20:27.067403+0000","last_unstale":"2024-09-11T00:22:56.601812+0000","last_undegraded":"2024-09-11T00:22:56.601812+0000","last_fullsized":"2024-09-11T00:22:56.601812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:29.439013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610162+0000","last_change":"2024-09-11T00:20:27.071586+0000","last_active":"2024-09-11T00:22:56.610162+0000","last_peered":"2024-09-11T00:22:56.610162+0000","last_clean":"2024-09-11T00:22:56.610162+0000","last_became_active":"2024-09-11T00:20:27.071072+0000","last_became_peered":"2024-09-11T00:20:27.071072+0000","last_unstale":"2024-09-11T00:22:56.610162+0000","last_undegraded":"2024-09-11T00:22:56.610162+0000","last_fullsized":"2024-09-11T00:22:56.610162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:34.133027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613642+0000","last_change":"2024-09-11T00:20:27.066683+0000","last_active":"2024-09-11T00:22:56.613642+0000","last_peered":"2024-09-11T00:22:56.613642+0000","last_clean":"2024-09-11T00:22:56.613642+0000","last_became_active":"2024-09-11T00:20:27.066355+0000","last_became_peered":"2024-09-11T00:20:27.066355+0000","last_unstale":"2024-09-11T00:22:56.613642+0000","last_undegraded":"2024-09-11T00:22:56.613642+0000","last_fullsized":"2024-09-11T00:22:56.613642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:26.868072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.598228+0000","last_change":"2024-09-11T00:20:27.060546+0000","last_active":"2024-09-11T00:22:56.598228+0000","last_peered":"2024-09-11T00:22:56.598228+0000","last_clean":"2024-09-11T00:22:56.598228+0000","last_became_active":"2024-09-11T00:20:27.059619+0000","last_became_peered":"2024-09-11T00:20:27.059619+0000","last_unstale":"2024-09-11T00:22:56.598228+0000","last_undegraded":"2024-09-11T00:22:56.598228+0000","last_fullsized":"2024-09-11T00:22:56.598228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.319854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607755+0000","last_change":"2024-09-11T00:20:27.073342+0000","last_active":"2024-09-11T00:22:56.607755+0000","last_peered":"2024-09-11T00:22:56.607755+0000","last_clean":"2024-09-11T00:22:56.607755+0000","last_became_active":"2024-09-11T00:20:27.072435+0000","last_became_peered":"2024-09-11T00:20:27.072435+0000","last_unstale":"2024-09-11T00:22:56.607755+0000","last_undegraded":"2024-09-11T00:22:56.607755+0000","last_fullsized":"2024-09-11T00:22:56.607755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:15.943515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612461+0000","last_change":"2024-09-11T00:20:27.063220+0000","last_active":"2024-09-11T00:22:56.612461+0000","last_peered":"2024-09-11T00:22:56.612461+0000","last_clean":"2024-09-11T00:22:56.612461+0000","last_became_active":"2024-09-11T00:20:27.062823+0000","last_became_peered":"2024-09-11T00:20:27.062823+0000","last_unstale":"2024-09-11T00:22:56.612461+0000","last_undegraded":"2024-09-11T00:22:56.612461+0000","last_fullsized":"2024-09-11T00:22:56.612461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:22.430295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613071+0000","last_change":"2024-09-11T00:20:27.062088+0000","last_active":"2024-09-11T00:22:56.613071+0000","last_peered":"2024-09-11T00:22:56.613071+0000","last_clean":"2024-09-11T00:22:56.613071+0000","last_became_active":"2024-09-11T00:20:27.061639+0000","last_became_peered":"2024-09-11T00:20:27.061639+0000","last_unstale":"2024-09-11T00:22:56.613071+0000","last_undegraded":"2024-09-11T00:22:56.613071+0000","last_fullsized":"2024-09-11T00:22:56.613071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.965908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611123+0000","last_change":"2024-09-11T00:20:27.067170+0000","last_active":"2024-09-11T00:22:56.611123+0000","last_peered":"2024-09-11T00:22:56.611123+0000","last_clean":"2024-09-11T00:22:56.611123+0000","last_became_active":"2024-09-11T00:20:27.066851+0000","last_became_peered":"2024-09-11T00:20:27.066851+0000","last_unstale":"2024-09-11T00:22:56.611123+0000","last_undegraded":"2024-09-11T00:22:56.611123+0000","last_fullsized":"2024-09-11T00:22:56.611123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:51.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604750+0000","last_change":"2024-09-11T00:21:28.233390+0000","last_active":"2024-09-11T00:22:56.604750+0000","last_peered":"2024-09-11T00:22:56.604750+0000","last_clean":"2024-09-11T00:22:56.604750+0000","last_became_active":"2024-09-11T00:21:28.232886+0000","last_became_peered":"2024-09-11T00:21:28.232886+0000","last_unstale":"2024-09-11T00:22:56.604750+0000","last_undegraded":"2024-09-11T00:22:56.604750+0000","last_fullsized":"2024-09-11T00:22:56.604750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:26.702069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606581+0000","last_change":"2024-09-11T00:20:27.059553+0000","last_active":"2024-09-11T00:22:56.606581+0000","last_peered":"2024-09-11T00:22:56.606581+0000","last_clean":"2024-09-11T00:22:56.606581+0000","last_became_active":"2024-09-11T00:20:27.058979+0000","last_became_peered":"2024-09-11T00:20:27.058979+0000","last_unstale":"2024-09-11T00:22:56.606581+0000","last_undegraded":"2024-09-11T00:22:56.606581+0000","last_fullsized":"2024-09-11T00:22:56.606581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:21.472828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606329+0000","last_change":"2024-09-11T00:22:28.831536+0000","last_active":"2024-09-11T00:22:56.606329+0000","last_peered":"2024-09-11T00:22:56.606329+0000","last_clean":"2024-09-11T00:22:56.606329+0000","last_became_active":"2024-09-11T00:22:28.831151+0000","last_became_peered":"2024-09-11T00:22:28.831151+0000","last_unstale":"2024-09-11T00:22:56.606329+0000","last_undegraded":"2024-09-11T00:22:56.606329+0000","last_fullsized":"2024-09-11T00:22:56.606329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.532570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.602775+0000","last_change":"2024-09-11T00:20:27.059140+0000","last_active":"2024-09-11T00:22:56.602775+0000","last_peered":"2024-09-11T00:22:56.602775+0000","last_clean":"2024-09-11T00:22:56.602775+0000","last_became_active":"2024-09-11T00:20:27.058765+0000","last_became_peered":"2024-09-11T00:20:27.058765+0000","last_unstale":"2024-09-11T00:22:56.602775+0000","last_undegraded":"2024-09-11T00:22:56.602775+0000","last_fullsized":"2024-09-11T00:22:56.602775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:43.947411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604652+0000","last_change":"2024-09-11T00:20:27.061149+0000","last_active":"2024-09-11T00:22:56.604652+0000","last_peered":"2024-09-11T00:22:56.604652+0000","last_clean":"2024-09-11T00:22:56.604652+0000","last_became_active":"2024-09-11T00:20:27.059949+0000","last_became_peered":"2024-09-11T00:20:27.059949+0000","last_unstale":"2024-09-11T00:22:56.604652+0000","last_undegraded":"2024-09-11T00:22:56.604652+0000","last_fullsized":"2024-09-11T00:22:56.604652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:03.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597666+0000","last_change":"2024-09-11T00:21:28.231322+0000","last_active":"2024-09-11T00:22:56.597666+0000","last_peered":"2024-09-11T00:22:56.597666+0000","last_clean":"2024-09-11T00:22:56.597666+0000","last_became_active":"2024-09-11T00:21:28.231075+0000","last_became_peered":"2024-09-11T00:21:28.231075+0000","last_unstale":"2024-09-11T00:22:56.597666+0000","last_undegraded":"2024-09-11T00:22:56.597666+0000","last_fullsized":"2024-09-11T00:22:56.597666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:50.887001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612406+0000","last_change":"2024-09-11T00:20:27.058175+0000","last_active":"2024-09-11T00:22:56.612406+0000","last_peered":"2024-09-11T00:22:56.612406+0000","last_clean":"2024-09-11T00:22:56.612406+0000","last_became_active":"2024-09-11T00:20:27.057797+0000","last_became_peered":"2024-09-11T00:20:27.057797+0000","last_unstale":"2024-09-11T00:22:56.612406+0000","last_undegraded":"2024-09-11T00:22:56.612406+0000","last_fullsized":"2024-09-11T00:22:56.612406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:29.370400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.609823+0000","last_change":"2024-09-11T00:20:27.062024+0000","last_active":"2024-09-11T00:22:56.609823+0000","last_peered":"2024-09-11T00:22:56.609823+0000","last_clean":"2024-09-11T00:22:56.609823+0000","last_became_active":"2024-09-11T00:20:27.061763+0000","last_became_peered":"2024-09-11T00:20:27.061763+0000","last_unstale":"2024-09-11T00:22:56.609823+0000","last_undegraded":"2024-09-11T00:22:56.609823+0000","last_fullsized":"2024-09-11T00:22:56.609823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:07.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597966+0000","last_change":"2024-09-11T00:20:27.060258+0000","last_active":"2024-09-11T00:22:56.597966+0000","last_peered":"2024-09-11T00:22:56.597966+0000","last_clean":"2024-09-11T00:22:56.597966+0000","last_became_active":"2024-09-11T00:20:27.059475+0000","last_became_peered":"2024-09-11T00:20:27.059475+0000","last_unstale":"2024-09-11T00:22:56.597966+0000","last_undegraded":"2024-09-11T00:22:56.597966+0000","last_fullsized":"2024-09-11T00:22:56.597966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:27.186322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.603092+0000","last_change":"2024-09-11T00:20:27.059730+0000","last_active":"2024-09-11T00:22:56.603092+0000","last_peered":"2024-09-11T00:22:56.603092+0000","last_clean":"2024-09-11T00:22:56.603092+0000","last_became_active":"2024-09-11T00:20:27.059389+0000","last_became_peered":"2024-09-11T00:20:27.059389+0000","last_unstale":"2024-09-11T00:22:56.603092+0000","last_undegraded":"2024-09-11T00:22:56.603092+0000","last_fullsized":"2024-09-11T00:22:56.603092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605141+0000","last_change":"2024-09-11T00:20:27.072765+0000","last_active":"2024-09-11T00:22:56.605141+0000","last_peered":"2024-09-11T00:22:56.605141+0000","last_clean":"2024-09-11T00:22:56.605141+0000","last_became_active":"2024-09-11T00:20:27.072392+0000","last_became_peered":"2024-09-11T00:20:27.072392+0000","last_unstale":"2024-09-11T00:22:56.605141+0000","last_undegraded":"2024-09-11T00:22:56.605141+0000","last_fullsized":"2024-09-11T00:22:56.605141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:27.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.600287+0000","last_change":"2024-09-11T00:20:27.056575+0000","last_active":"2024-09-11T00:22:56.600287+0000","last_peered":"2024-09-11T00:22:56.600287+0000","last_clean":"2024-09-11T00:22:56.600287+0000","last_became_active":"2024-09-11T00:20:27.056264+0000","last_became_peered":"2024-09-11T00:20:27.056264+0000","last_unstale":"2024-09-11T00:22:56.600287+0000","last_undegraded":"2024-09-11T00:22:56.600287+0000","last_fullsized":"2024-09-11T00:22:56.600287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:39.722518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610079+0000","last_change":"2024-09-11T00:20:27.065153+0000","last_active":"2024-09-11T00:22:56.610079+0000","last_peered":"2024-09-11T00:22:56.610079+0000","last_clean":"2024-09-11T00:22:56.610079+0000","last_became_active":"2024-09-11T00:20:27.062978+0000","last_became_peered":"2024-09-11T00:20:27.062978+0000","last_unstale":"2024-09-11T00:22:56.610079+0000","last_undegraded":"2024-09-11T00:22:56.610079+0000","last_fullsized":"2024-09-11T00:22:56.610079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:37.820467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605823+0000","last_change":"2024-09-11T00:20:27.070085+0000","last_active":"2024-09-11T00:22:56.605823+0000","last_peered":"2024-09-11T00:22:56.605823+0000","last_clean":"2024-09-11T00:22:56.605823+0000","last_became_active":"2024-09-11T00:20:27.069809+0000","last_became_peered":"2024-09-11T00:20:27.069809+0000","last_unstale":"2024-09-11T00:22:56.605823+0000","last_undegraded":"2024-09-11T00:22:56.605823+0000","last_fullsized":"2024-09-11T00:22:56.605823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:01.048590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612973+0000","last_change":"2024-09-11T00:20:27.064584+0000","last_active":"2024-09-11T00:22:56.612973+0000","last_peered":"2024-09-11T00:22:56.612973+0000","last_clean":"2024-09-11T00:22:56.612973+0000","last_became_active":"2024-09-11T00:20:27.064271+0000","last_became_peered":"2024-09-11T00:20:27.064271+0000","last_unstale":"2024-09-11T00:22:56.612973+0000","last_undegraded":"2024-09-11T00:22:56.612973+0000","last_fullsized":"2024-09-11T00:22:56.612973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:53.717744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67085,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":777,"num_read_kb":461,"num_write":182,"num_write_kb":127,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":206317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710771,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":515118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906999,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1716224,"data_stored":1165217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070526,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266755,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":515050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462984,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1111523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":31914445},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":510418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822740,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986266,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":515379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":510612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":574531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":578333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":511806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:23:33.790 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T00:23:33.791 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T00:23:34.166 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:34 smithi103 ceph-mon[30578]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:34.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:34 smithi117 ceph-mon[35525]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:34.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:34 smithi183 ceph-mon[35934]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:34.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 -- 172.21.15.103:0/572286069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 msgr2=0x7ff38810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:34.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 --2- 172.21.15.103:0/572286069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 0x7ff38810f930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff380009ca0 tx=0x7ff38002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 -- 172.21.15.103:0/572286069 shutdown_connections 2024-09-11T00:23:34.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 --2- 172.21.15.103:0/572286069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 0x7ff38810f930 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 --2- 172.21.15.103:0/572286069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff388105570 0x7ff3881059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 --2- 172.21.15.103:0/572286069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 0x7ff388101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 -- 172.21.15.103:0/572286069 >> 172.21.15.103:0/572286069 conn(0x7ff388078100 msgr2=0x7ff3880ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:34.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 -- 172.21.15.103:0/572286069 shutdown_connections 2024-09-11T00:23:34.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.787+0000 7ff38cc00640 1 -- 172.21.15.103:0/572286069 wait complete. 2024-09-11T00:23:34.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.788+0000 7ff38cc00640 1 Processor -- start 2024-09-11T00:23:34.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.788+0000 7ff38cc00640 1 -- start start 2024-09-11T00:23:34.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff38cc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 0x7ff388199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff38cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff388105570 0x7ff388199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff38cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 0x7ff38819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff38cc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff388112df0 con 0x7ff388105ef0 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff38cc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ff388112c70 con 0x7ff388101220 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff38cc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff388112f70 con 0x7ff388105570 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff385e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 0x7ff388199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff386800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 0x7ff38819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff385e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 0x7ff388199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:36958/0 (socket says 172.21.15.103:36958) 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff385e00640 1 -- 172.21.15.103:0/1715557461 learned_addr learned my addr 172.21.15.103:0/1715557461 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:34.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.789+0000 7ff385400640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff388105570 0x7ff388199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff385e00640 1 -- 172.21.15.103:0/1715557461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff388105570 msgr2=0x7ff388199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff385e00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff388105570 0x7ff388199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff385e00640 1 -- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 msgr2=0x7ff38819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff385e00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 0x7ff38819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff385e00640 1 -- 172.21.15.103:0/1715557461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38819f2d0 con 0x7ff388101220 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff386800640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 0x7ff38819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff385e00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 0x7ff388199220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff37800e9f0 tx=0x7ff37800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:34.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff37ea00640 1 -- 172.21.15.103:0/1715557461 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37800cd80 con 0x7ff388101220 2024-09-11T00:23:34.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff37ea00640 1 -- 172.21.15.103:0/1715557461 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3780108e0 con 0x7ff388101220 2024-09-11T00:23:34.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38819f5c0 con 0x7ff388101220 2024-09-11T00:23:34.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff37ea00640 1 -- 172.21.15.103:0/1715557461 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378018980 con 0x7ff388101220 2024-09-11T00:23:34.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.790+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3881a5c80 con 0x7ff388101220 2024-09-11T00:23:34.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.791+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff388101600 con 0x7ff388101220 2024-09-11T00:23:34.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.794+0000 7ff37ea00640 1 -- 172.21.15.103:0/1715557461 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ff378010430 con 0x7ff388101220 2024-09-11T00:23:34.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.795+0000 7ff37ea00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff36c0b52e0 0x7ff36c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:34.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.796+0000 7ff37ea00640 1 -- 172.21.15.103:0/1715557461 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff3780e67c0 con 0x7ff388101220 2024-09-11T00:23:34.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.796+0000 7ff385400640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff36c0b52e0 0x7ff36c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:34.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.796+0000 7ff385400640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff36c0b52e0 0x7ff36c0b77a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff388107da0 tx=0x7ff37400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:34.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.797+0000 7ff37ea00640 1 -- 172.21.15.103:0/1715557461 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff3780acc40 con 0x7ff388101220 2024-09-11T00:23:34.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.977+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 --> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff388108eb0 con 0x7ff36c0b52e0 2024-09-11T00:23:34.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.982+0000 7ff37ea00640 1 -- 172.21.15.103:0/1715557461 <== mgr.14150 v2:172.21.15.103:6800/3157485146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340477 (secure 0 0 0) 0x7ff388108eb0 con 0x7ff36c0b52e0 2024-09-11T00:23:34.988 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:34.989 INFO:teuthology.orchestra.run.smithi103.stderr:dumped all 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff36c0b52e0 msgr2=0x7ff36c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff36c0b52e0 0x7ff36c0b77a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff388107da0 tx=0x7ff37400b410 comp rx=0 tx=0).stop 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 msgr2=0x7ff388199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 0x7ff388199220 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff37800e9f0 tx=0x7ff37800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 shutdown_connections 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ff36c0b52e0 0x7ff36c0b77a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff388105ef0 0x7ff38819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff388105570 0x7ff388199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 --2- 172.21.15.103:0/1715557461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff388101220 0x7ff388199220 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:34.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 >> 172.21.15.103:0/1715557461 conn(0x7ff388078100 msgr2=0x7ff388106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:34.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 shutdown_connections 2024-09-11T00:23:34.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:34.987+0000 7ff38cc00640 1 -- 172.21.15.103:0/1715557461 wait complete. 2024-09-11T00:23:35.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:35 smithi103 ceph-mon[30578]: pgmap v413: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:23:35.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:35 smithi117 ceph-mon[35525]: pgmap v413: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:23:35.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:35 smithi183 ceph-mon[35934]: pgmap v413: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:23:36.071 INFO:teuthology.orchestra.run.smithi103.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-11T00:23:34.408348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663060,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":879,"num_read_kb":541,"num_write":308,"num_write_kb":2285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":2951952,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":750,"ondisk_log_size":750,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375836,"kb_used_data":14108,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565988,"statfs":{"total":1151940427776,"available":1151555571712,"internally_reserved":0,"allocated":14446592,"data_stored":8145182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":370193724},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5995,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":48,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003157"},"pg_stats":[{"pgid":"2.1f","version":"85'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680924+0000","last_change":"2024-09-11T00:20:24.027578+0000","last_active":"2024-09-11T00:22:28.680924+0000","last_peered":"2024-09-11T00:22:28.680924+0000","last_clean":"2024-09-11T00:22:28.680924+0000","last_became_active":"2024-09-11T00:20:24.027174+0000","last_became_peered":"2024-09-11T00:20:24.027174+0000","last_unstale":"2024-09-11T00:22:28.680924+0000","last_undegraded":"2024-09-11T00:22:28.680924+0000","last_fullsized":"2024-09-11T00:22:28.680924+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:45:15.429053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.595270+0000","last_change":"2024-09-11T00:20:27.064047+0000","last_active":"2024-09-11T00:22:56.595270+0000","last_peered":"2024-09-11T00:22:56.595270+0000","last_clean":"2024-09-11T00:22:56.595270+0000","last_became_active":"2024-09-11T00:20:27.063756+0000","last_became_peered":"2024-09-11T00:20:27.063756+0000","last_unstale":"2024-09-11T00:22:56.595270+0000","last_undegraded":"2024-09-11T00:22:56.595270+0000","last_fullsized":"2024-09-11T00:22:56.595270+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:20.695888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679261+0000","last_change":"2024-09-11T00:20:37.449428+0000","last_active":"2024-09-11T00:22:28.679261+0000","last_peered":"2024-09-11T00:22:28.679261+0000","last_clean":"2024-09-11T00:22:28.679261+0000","last_became_active":"2024-09-11T00:20:37.449015+0000","last_became_peered":"2024-09-11T00:20:37.449015+0000","last_unstale":"2024-09-11T00:22:28.679261+0000","last_undegraded":"2024-09-11T00:22:28.679261+0000","last_fullsized":"2024-09-11T00:22:28.679261+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:42.983990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864076+0000","last_change":"2024-09-11T00:20:24.035966+0000","last_active":"2024-09-11T00:22:28.864076+0000","last_peered":"2024-09-11T00:22:28.864076+0000","last_clean":"2024-09-11T00:22:28.864076+0000","last_became_active":"2024-09-11T00:20:24.035663+0000","last_became_peered":"2024-09-11T00:20:24.035663+0000","last_unstale":"2024-09-11T00:22:28.864076+0000","last_undegraded":"2024-09-11T00:22:28.864076+0000","last_fullsized":"2024-09-11T00:22:28.864076+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:22.529392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612592+0000","last_change":"2024-09-11T00:20:27.066470+0000","last_active":"2024-09-11T00:22:56.612592+0000","last_peered":"2024-09-11T00:22:56.612592+0000","last_clean":"2024-09-11T00:22:56.612592+0000","last_became_active":"2024-09-11T00:20:27.065424+0000","last_became_peered":"2024-09-11T00:20:27.065424+0000","last_unstale":"2024-09-11T00:22:56.612592+0000","last_undegraded":"2024-09-11T00:22:56.612592+0000","last_fullsized":"2024-09-11T00:22:56.612592+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:39.967332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864816+0000","last_change":"2024-09-11T00:20:37.453105+0000","last_active":"2024-09-11T00:22:28.864816+0000","last_peered":"2024-09-11T00:22:28.864816+0000","last_clean":"2024-09-11T00:22:28.864816+0000","last_became_active":"2024-09-11T00:20:37.452319+0000","last_became_peered":"2024-09-11T00:20:37.452319+0000","last_unstale":"2024-09-11T00:22:28.864816+0000","last_undegraded":"2024-09-11T00:22:28.864816+0000","last_fullsized":"2024-09-11T00:22:28.864816+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:05.873046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864807+0000","last_change":"2024-09-11T00:20:24.022277+0000","last_active":"2024-09-11T00:22:28.864807+0000","last_peered":"2024-09-11T00:22:28.864807+0000","last_clean":"2024-09-11T00:22:28.864807+0000","last_became_active":"2024-09-11T00:20:24.021893+0000","last_became_peered":"2024-09-11T00:20:24.021893+0000","last_unstale":"2024-09-11T00:22:28.864807+0000","last_undegraded":"2024-09-11T00:22:28.864807+0000","last_fullsized":"2024-09-11T00:22:28.864807+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.994032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613024+0000","last_change":"2024-09-11T00:20:27.063368+0000","last_active":"2024-09-11T00:22:56.613024+0000","last_peered":"2024-09-11T00:22:56.613024+0000","last_clean":"2024-09-11T00:22:56.613024+0000","last_became_active":"2024-09-11T00:20:27.063051+0000","last_became_peered":"2024-09-11T00:20:27.063051+0000","last_unstale":"2024-09-11T00:22:56.613024+0000","last_undegraded":"2024-09-11T00:22:56.613024+0000","last_fullsized":"2024-09-11T00:22:56.613024+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:16:09.815971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864842+0000","last_change":"2024-09-11T00:21:28.229073+0000","last_active":"2024-09-11T00:22:28.864842+0000","last_peered":"2024-09-11T00:22:28.864842+0000","last_clean":"2024-09-11T00:22:28.864842+0000","last_became_active":"2024-09-11T00:21:28.228775+0000","last_became_peered":"2024-09-11T00:21:28.228775+0000","last_unstale":"2024-09-11T00:22:28.864842+0000","last_undegraded":"2024-09-11T00:22:28.864842+0000","last_fullsized":"2024-09-11T00:22:28.864842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:28:03.886727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864759+0000","last_change":"2024-09-11T00:20:24.022562+0000","last_active":"2024-09-11T00:22:28.864759+0000","last_peered":"2024-09-11T00:22:28.864759+0000","last_clean":"2024-09-11T00:22:28.864759+0000","last_became_active":"2024-09-11T00:20:24.022207+0000","last_became_peered":"2024-09-11T00:20:24.022207+0000","last_unstale":"2024-09-11T00:22:28.864759+0000","last_undegraded":"2024-09-11T00:22:28.864759+0000","last_fullsized":"2024-09-11T00:22:28.864759+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:01.071034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608323+0000","last_change":"2024-09-11T00:20:27.062572+0000","last_active":"2024-09-11T00:22:56.608323+0000","last_peered":"2024-09-11T00:22:56.608323+0000","last_clean":"2024-09-11T00:22:56.608323+0000","last_became_active":"2024-09-11T00:20:27.062288+0000","last_became_peered":"2024-09-11T00:20:27.062288+0000","last_unstale":"2024-09-11T00:22:56.608323+0000","last_undegraded":"2024-09-11T00:22:56.608323+0000","last_fullsized":"2024-09-11T00:22:56.608323+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:37.391822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864680+0000","last_change":"2024-09-11T00:20:37.444028+0000","last_active":"2024-09-11T00:22:28.864680+0000","last_peered":"2024-09-11T00:22:28.864680+0000","last_clean":"2024-09-11T00:22:28.864680+0000","last_became_active":"2024-09-11T00:20:37.443344+0000","last_became_peered":"2024-09-11T00:20:37.443344+0000","last_unstale":"2024-09-11T00:22:28.864680+0000","last_undegraded":"2024-09-11T00:22:28.864680+0000","last_fullsized":"2024-09-11T00:22:28.864680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:17.347054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611869+0000","last_change":"2024-09-11T00:20:27.060317+0000","last_active":"2024-09-11T00:22:56.611869+0000","last_peered":"2024-09-11T00:22:56.611869+0000","last_clean":"2024-09-11T00:22:56.611869+0000","last_became_active":"2024-09-11T00:20:27.059983+0000","last_became_peered":"2024-09-11T00:20:27.059983+0000","last_unstale":"2024-09-11T00:22:56.611869+0000","last_undegraded":"2024-09-11T00:22:56.611869+0000","last_fullsized":"2024-09-11T00:22:56.611869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:37:28.405553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864258+0000","last_change":"2024-09-11T00:20:24.025555+0000","last_active":"2024-09-11T00:22:28.864258+0000","last_peered":"2024-09-11T00:22:28.864258+0000","last_clean":"2024-09-11T00:22:28.864258+0000","last_became_active":"2024-09-11T00:20:24.025171+0000","last_became_peered":"2024-09-11T00:20:24.025171+0000","last_unstale":"2024-09-11T00:22:28.864258+0000","last_undegraded":"2024-09-11T00:22:28.864258+0000","last_fullsized":"2024-09-11T00:22:28.864258+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:22.708985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864496+0000","last_change":"2024-09-11T00:20:37.442057+0000","last_active":"2024-09-11T00:22:28.864496+0000","last_peered":"2024-09-11T00:22:28.864496+0000","last_clean":"2024-09-11T00:22:28.864496+0000","last_became_active":"2024-09-11T00:20:37.441555+0000","last_became_peered":"2024-09-11T00:20:37.441555+0000","last_unstale":"2024-09-11T00:22:28.864496+0000","last_undegraded":"2024-09-11T00:22:28.864496+0000","last_fullsized":"2024-09-11T00:22:28.864496+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:08.155763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611677+0000","last_change":"2024-09-11T00:20:27.071533+0000","last_active":"2024-09-11T00:22:56.611677+0000","last_peered":"2024-09-11T00:22:56.611677+0000","last_clean":"2024-09-11T00:22:56.611677+0000","last_became_active":"2024-09-11T00:20:27.068771+0000","last_became_peered":"2024-09-11T00:20:27.068771+0000","last_unstale":"2024-09-11T00:22:56.611677+0000","last_undegraded":"2024-09-11T00:22:56.611677+0000","last_fullsized":"2024-09-11T00:22:56.611677+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:05.471476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679914+0000","last_change":"2024-09-11T00:20:24.023282+0000","last_active":"2024-09-11T00:22:28.679914+0000","last_peered":"2024-09-11T00:22:28.679914+0000","last_clean":"2024-09-11T00:22:28.679914+0000","last_became_active":"2024-09-11T00:20:24.022914+0000","last_became_peered":"2024-09-11T00:20:24.022914+0000","last_unstale":"2024-09-11T00:22:28.679914+0000","last_undegraded":"2024-09-11T00:22:28.679914+0000","last_fullsized":"2024-09-11T00:22:28.679914+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:45.231733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864811+0000","last_change":"2024-09-11T00:20:37.457338+0000","last_active":"2024-09-11T00:22:28.864811+0000","last_peered":"2024-09-11T00:22:28.864811+0000","last_clean":"2024-09-11T00:22:28.864811+0000","last_became_active":"2024-09-11T00:20:37.456993+0000","last_became_peered":"2024-09-11T00:20:37.456993+0000","last_unstale":"2024-09-11T00:22:28.864811+0000","last_undegraded":"2024-09-11T00:22:28.864811+0000","last_fullsized":"2024-09-11T00:22:28.864811+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:15:57.284412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606431+0000","last_change":"2024-09-11T00:20:27.062752+0000","last_active":"2024-09-11T00:22:56.606431+0000","last_peered":"2024-09-11T00:22:56.606431+0000","last_clean":"2024-09-11T00:22:56.606431+0000","last_became_active":"2024-09-11T00:20:27.061569+0000","last_became_peered":"2024-09-11T00:20:27.061569+0000","last_unstale":"2024-09-11T00:22:56.606431+0000","last_undegraded":"2024-09-11T00:22:56.606431+0000","last_fullsized":"2024-09-11T00:22:56.606431+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:30.904423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864019+0000","last_change":"2024-09-11T00:20:24.023197+0000","last_active":"2024-09-11T00:22:28.864019+0000","last_peered":"2024-09-11T00:22:28.864019+0000","last_clean":"2024-09-11T00:22:28.864019+0000","last_became_active":"2024-09-11T00:20:24.022867+0000","last_became_peered":"2024-09-11T00:20:24.022867+0000","last_unstale":"2024-09-11T00:22:28.864019+0000","last_undegraded":"2024-09-11T00:22:28.864019+0000","last_fullsized":"2024-09-11T00:22:28.864019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:37.949437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680695+0000","last_change":"2024-09-11T00:20:37.451731+0000","last_active":"2024-09-11T00:22:28.680695+0000","last_peered":"2024-09-11T00:22:28.680695+0000","last_clean":"2024-09-11T00:22:28.680695+0000","last_became_active":"2024-09-11T00:20:37.447710+0000","last_became_peered":"2024-09-11T00:20:37.447710+0000","last_unstale":"2024-09-11T00:22:28.680695+0000","last_undegraded":"2024-09-11T00:22:28.680695+0000","last_fullsized":"2024-09-11T00:22:28.680695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:11:38.471216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.565404+0000","last_change":"2024-09-11T00:20:27.066021+0000","last_active":"2024-09-11T00:22:56.565404+0000","last_peered":"2024-09-11T00:22:56.565404+0000","last_clean":"2024-09-11T00:22:56.565404+0000","last_became_active":"2024-09-11T00:20:27.065652+0000","last_became_peered":"2024-09-11T00:20:27.065652+0000","last_unstale":"2024-09-11T00:22:56.565404+0000","last_undegraded":"2024-09-11T00:22:56.565404+0000","last_fullsized":"2024-09-11T00:22:56.565404+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:13:01.933031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864139+0000","last_change":"2024-09-11T00:20:24.035619+0000","last_active":"2024-09-11T00:22:28.864139+0000","last_peered":"2024-09-11T00:22:28.864139+0000","last_clean":"2024-09-11T00:22:28.864139+0000","last_became_active":"2024-09-11T00:20:24.035358+0000","last_became_peered":"2024-09-11T00:20:24.035358+0000","last_unstale":"2024-09-11T00:22:28.864139+0000","last_undegraded":"2024-09-11T00:22:28.864139+0000","last_fullsized":"2024-09-11T00:22:28.864139+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:56.890058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865646+0000","last_change":"2024-09-11T00:21:28.226415+0000","last_active":"2024-09-11T00:22:28.865646+0000","last_peered":"2024-09-11T00:22:28.865646+0000","last_clean":"2024-09-11T00:22:28.865646+0000","last_became_active":"2024-09-11T00:21:28.225894+0000","last_became_peered":"2024-09-11T00:21:28.225894+0000","last_unstale":"2024-09-11T00:22:28.865646+0000","last_undegraded":"2024-09-11T00:22:28.865646+0000","last_fullsized":"2024-09-11T00:22:28.865646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:59.618835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,10,9],"acting":[4,8,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606794+0000","last_change":"2024-09-11T00:22:28.866891+0000","last_active":"2024-09-11T00:22:56.606794+0000","last_peered":"2024-09-11T00:22:56.606794+0000","last_clean":"2024-09-11T00:22:56.606794+0000","last_became_active":"2024-09-11T00:22:28.866541+0000","last_became_peered":"2024-09-11T00:22:28.866541+0000","last_unstale":"2024-09-11T00:22:56.606794+0000","last_undegraded":"2024-09-11T00:22:56.606794+0000","last_fullsized":"2024-09-11T00:22:56.606794+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:35.331709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865567+0000","last_change":"2024-09-11T00:20:24.535701+0000","last_active":"2024-09-11T00:22:28.865567+0000","last_peered":"2024-09-11T00:22:28.865567+0000","last_clean":"2024-09-11T00:22:28.865567+0000","last_became_active":"2024-09-11T00:20:24.535389+0000","last_became_peered":"2024-09-11T00:20:24.535389+0000","last_unstale":"2024-09-11T00:22:28.865567+0000","last_undegraded":"2024-09-11T00:22:28.865567+0000","last_fullsized":"2024-09-11T00:22:28.865567+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:20:29.044747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679639+0000","last_change":"2024-09-11T00:20:37.435584+0000","last_active":"2024-09-11T00:22:28.679639+0000","last_peered":"2024-09-11T00:22:28.679639+0000","last_clean":"2024-09-11T00:22:28.679639+0000","last_became_active":"2024-09-11T00:20:37.435237+0000","last_became_peered":"2024-09-11T00:20:37.435237+0000","last_unstale":"2024-09-11T00:22:28.679639+0000","last_undegraded":"2024-09-11T00:22:28.679639+0000","last_fullsized":"2024-09-11T00:22:28.679639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:49:34.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610308+0000","last_change":"2024-09-11T00:20:27.068718+0000","last_active":"2024-09-11T00:22:56.610308+0000","last_peered":"2024-09-11T00:22:56.610308+0000","last_clean":"2024-09-11T00:22:56.610308+0000","last_became_active":"2024-09-11T00:20:27.058921+0000","last_became_peered":"2024-09-11T00:20:27.058921+0000","last_unstale":"2024-09-11T00:22:56.610308+0000","last_undegraded":"2024-09-11T00:22:56.610308+0000","last_fullsized":"2024-09-11T00:22:56.610308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:33:16.056742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":59,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:57.080412+0000","last_change":"2024-09-11T00:20:24.023580+0000","last_active":"2024-09-11T00:22:57.080412+0000","last_peered":"2024-09-11T00:22:57.080412+0000","last_clean":"2024-09-11T00:22:57.080412+0000","last_became_active":"2024-09-11T00:20:24.023029+0000","last_became_peered":"2024-09-11T00:20:24.023029+0000","last_unstale":"2024-09-11T00:22:57.080412+0000","last_undegraded":"2024-09-11T00:22:57.080412+0000","last_fullsized":"2024-09-11T00:22:57.080412+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:05.574228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863468+0000","last_change":"2024-09-11T00:20:37.449229+0000","last_active":"2024-09-11T00:22:28.863468+0000","last_peered":"2024-09-11T00:22:28.863468+0000","last_clean":"2024-09-11T00:22:28.863468+0000","last_became_active":"2024-09-11T00:20:37.447464+0000","last_became_peered":"2024-09-11T00:20:37.447464+0000","last_unstale":"2024-09-11T00:22:28.863468+0000","last_undegraded":"2024-09-11T00:22:28.863468+0000","last_fullsized":"2024-09-11T00:22:28.863468+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:47.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597838+0000","last_change":"2024-09-11T00:20:27.063518+0000","last_active":"2024-09-11T00:22:56.597838+0000","last_peered":"2024-09-11T00:22:56.597838+0000","last_clean":"2024-09-11T00:22:56.597838+0000","last_became_active":"2024-09-11T00:20:27.063122+0000","last_became_peered":"2024-09-11T00:20:27.063122+0000","last_unstale":"2024-09-11T00:22:56.597838+0000","last_undegraded":"2024-09-11T00:22:56.597838+0000","last_fullsized":"2024-09-11T00:22:56.597838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:52.269862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864785+0000","last_change":"2024-09-11T00:20:24.035835+0000","last_active":"2024-09-11T00:22:28.864785+0000","last_peered":"2024-09-11T00:22:28.864785+0000","last_clean":"2024-09-11T00:22:28.864785+0000","last_became_active":"2024-09-11T00:20:24.035471+0000","last_became_peered":"2024-09-11T00:20:24.035471+0000","last_unstale":"2024-09-11T00:22:28.864785+0000","last_undegraded":"2024-09-11T00:22:28.864785+0000","last_fullsized":"2024-09-11T00:22:28.864785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:37.114863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863908+0000","last_change":"2024-09-11T00:20:37.439609+0000","last_active":"2024-09-11T00:22:28.863908+0000","last_peered":"2024-09-11T00:22:28.863908+0000","last_clean":"2024-09-11T00:22:28.863908+0000","last_became_active":"2024-09-11T00:20:37.439139+0000","last_became_peered":"2024-09-11T00:20:37.439139+0000","last_unstale":"2024-09-11T00:22:28.863908+0000","last_undegraded":"2024-09-11T00:22:28.863908+0000","last_fullsized":"2024-09-11T00:22:28.863908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:19.152096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610140+0000","last_change":"2024-09-11T00:20:27.062879+0000","last_active":"2024-09-11T00:22:56.610140+0000","last_peered":"2024-09-11T00:22:56.610140+0000","last_clean":"2024-09-11T00:22:56.610140+0000","last_became_active":"2024-09-11T00:20:27.062200+0000","last_became_peered":"2024-09-11T00:20:27.062200+0000","last_unstale":"2024-09-11T00:22:56.610140+0000","last_undegraded":"2024-09-11T00:22:56.610140+0000","last_fullsized":"2024-09-11T00:22:56.610140+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:57.308376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680751+0000","last_change":"2024-09-11T00:21:28.230802+0000","last_active":"2024-09-11T00:22:28.680751+0000","last_peered":"2024-09-11T00:22:28.680751+0000","last_clean":"2024-09-11T00:22:28.680751+0000","last_became_active":"2024-09-11T00:21:28.230378+0000","last_became_peered":"2024-09-11T00:21:28.230378+0000","last_unstale":"2024-09-11T00:22:28.680751+0000","last_undegraded":"2024-09-11T00:22:28.680751+0000","last_fullsized":"2024-09-11T00:22:28.680751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:35.676964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864428+0000","last_change":"2024-09-11T00:20:37.444912+0000","last_active":"2024-09-11T00:22:28.864428+0000","last_peered":"2024-09-11T00:22:28.864428+0000","last_clean":"2024-09-11T00:22:28.864428+0000","last_became_active":"2024-09-11T00:20:37.444472+0000","last_became_peered":"2024-09-11T00:20:37.444472+0000","last_unstale":"2024-09-11T00:22:28.864428+0000","last_undegraded":"2024-09-11T00:22:28.864428+0000","last_fullsized":"2024-09-11T00:22:28.864428+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:23.263981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606705+0000","last_change":"2024-09-11T00:20:27.060477+0000","last_active":"2024-09-11T00:22:56.606705+0000","last_peered":"2024-09-11T00:22:56.606705+0000","last_clean":"2024-09-11T00:22:56.606705+0000","last_became_active":"2024-09-11T00:20:27.060128+0000","last_became_peered":"2024-09-11T00:20:27.060128+0000","last_unstale":"2024-09-11T00:22:56.606705+0000","last_undegraded":"2024-09-11T00:22:56.606705+0000","last_fullsized":"2024-09-11T00:22:56.606705+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:55.967799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866156+0000","last_change":"2024-09-11T00:20:24.028229+0000","last_active":"2024-09-11T00:22:28.866156+0000","last_peered":"2024-09-11T00:22:28.866156+0000","last_clean":"2024-09-11T00:22:28.866156+0000","last_became_active":"2024-09-11T00:20:24.027832+0000","last_became_peered":"2024-09-11T00:20:24.027832+0000","last_unstale":"2024-09-11T00:22:28.866156+0000","last_undegraded":"2024-09-11T00:22:28.866156+0000","last_fullsized":"2024-09-11T00:22:28.866156+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:19.960403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682873+0000","last_change":"2024-09-11T00:20:37.442414+0000","last_active":"2024-09-11T00:22:28.682873+0000","last_peered":"2024-09-11T00:22:28.682873+0000","last_clean":"2024-09-11T00:22:28.682873+0000","last_became_active":"2024-09-11T00:20:37.442021+0000","last_became_peered":"2024-09-11T00:20:37.442021+0000","last_unstale":"2024-09-11T00:22:28.682873+0000","last_undegraded":"2024-09-11T00:22:28.682873+0000","last_fullsized":"2024-09-11T00:22:28.682873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:21.296638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.592369+0000","last_change":"2024-09-11T00:21:28.233303+0000","last_active":"2024-09-11T00:22:56.592369+0000","last_peered":"2024-09-11T00:22:56.592369+0000","last_clean":"2024-09-11T00:22:56.592369+0000","last_became_active":"2024-09-11T00:21:28.232039+0000","last_became_peered":"2024-09-11T00:21:28.232039+0000","last_unstale":"2024-09-11T00:22:56.592369+0000","last_undegraded":"2024-09-11T00:22:56.592369+0000","last_fullsized":"2024-09-11T00:22:56.592369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:02:36.828102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":176,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:29.403808+0000","last_change":"2024-09-11T00:20:24.037218+0000","last_active":"2024-09-11T00:23:29.403808+0000","last_peered":"2024-09-11T00:23:29.403808+0000","last_clean":"2024-09-11T00:23:29.403808+0000","last_became_active":"2024-09-11T00:20:24.036005+0000","last_became_peered":"2024-09-11T00:20:24.036005+0000","last_unstale":"2024-09-11T00:23:29.403808+0000","last_undegraded":"2024-09-11T00:23:29.403808+0000","last_fullsized":"2024-09-11T00:23:29.403808+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:08:17.758812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683067+0000","last_change":"2024-09-11T00:20:37.445584+0000","last_active":"2024-09-11T00:22:28.683067+0000","last_peered":"2024-09-11T00:22:28.683067+0000","last_clean":"2024-09-11T00:22:28.683067+0000","last_became_active":"2024-09-11T00:20:37.445196+0000","last_became_peered":"2024-09-11T00:20:37.445196+0000","last_unstale":"2024-09-11T00:22:28.683067+0000","last_undegraded":"2024-09-11T00:22:28.683067+0000","last_fullsized":"2024-09-11T00:22:28.683067+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:05.966282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.598168+0000","last_change":"2024-09-11T00:20:27.060428+0000","last_active":"2024-09-11T00:22:56.598168+0000","last_peered":"2024-09-11T00:22:56.598168+0000","last_clean":"2024-09-11T00:22:56.598168+0000","last_became_active":"2024-09-11T00:20:27.060057+0000","last_became_peered":"2024-09-11T00:20:27.060057+0000","last_unstale":"2024-09-11T00:22:56.598168+0000","last_undegraded":"2024-09-11T00:22:56.598168+0000","last_fullsized":"2024-09-11T00:22:56.598168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:37.027054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865091+0000","last_change":"2024-09-11T00:20:24.025496+0000","last_active":"2024-09-11T00:22:28.865091+0000","last_peered":"2024-09-11T00:22:28.865091+0000","last_clean":"2024-09-11T00:22:28.865091+0000","last_became_active":"2024-09-11T00:20:24.025158+0000","last_became_peered":"2024-09-11T00:20:24.025158+0000","last_unstale":"2024-09-11T00:22:28.865091+0000","last_undegraded":"2024-09-11T00:22:28.865091+0000","last_fullsized":"2024-09-11T00:22:28.865091+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:40.439708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734460+0000","last_change":"2024-09-11T00:20:37.453118+0000","last_active":"2024-09-11T00:22:28.734460+0000","last_peered":"2024-09-11T00:22:28.734460+0000","last_clean":"2024-09-11T00:22:28.734460+0000","last_became_active":"2024-09-11T00:20:37.452669+0000","last_became_peered":"2024-09-11T00:20:37.452669+0000","last_unstale":"2024-09-11T00:22:28.734460+0000","last_undegraded":"2024-09-11T00:22:28.734460+0000","last_fullsized":"2024-09-11T00:22:28.734460+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:51:58.294897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606977+0000","last_change":"2024-09-11T00:20:27.069708+0000","last_active":"2024-09-11T00:22:56.606977+0000","last_peered":"2024-09-11T00:22:56.606977+0000","last_clean":"2024-09-11T00:22:56.606977+0000","last_became_active":"2024-09-11T00:20:27.067500+0000","last_became_peered":"2024-09-11T00:20:27.067500+0000","last_unstale":"2024-09-11T00:22:56.606977+0000","last_undegraded":"2024-09-11T00:22:56.606977+0000","last_fullsized":"2024-09-11T00:22:56.606977+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:11:57.084037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864981+0000","last_change":"2024-09-11T00:20:24.023504+0000","last_active":"2024-09-11T00:22:28.864981+0000","last_peered":"2024-09-11T00:22:28.864981+0000","last_clean":"2024-09-11T00:22:28.864981+0000","last_became_active":"2024-09-11T00:20:24.022706+0000","last_became_peered":"2024-09-11T00:20:24.022706+0000","last_unstale":"2024-09-11T00:22:28.864981+0000","last_undegraded":"2024-09-11T00:22:28.864981+0000","last_fullsized":"2024-09-11T00:22:28.864981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:02:21.216325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":120,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865351+0000","last_change":"2024-09-11T00:19:05.531191+0000","last_active":"2024-09-11T00:22:28.865351+0000","last_peered":"2024-09-11T00:22:28.865351+0000","last_clean":"2024-09-11T00:22:28.865351+0000","last_became_active":"2024-09-11T00:19:05.525664+0000","last_became_peered":"2024-09-11T00:19:05.525664+0000","last_unstale":"2024-09-11T00:22:28.865351+0000","last_undegraded":"2024-09-11T00:22:28.865351+0000","last_fullsized":"2024-09-11T00:22:28.865351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:15:23.365345+0000","last_clean_scrub_stamp":"2024-09-11T00:15:23.365345+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:06.253537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.866037+0000","last_change":"2024-09-11T00:20:37.446403+0000","last_active":"2024-09-11T00:22:28.866037+0000","last_peered":"2024-09-11T00:22:28.866037+0000","last_clean":"2024-09-11T00:22:28.866037+0000","last_became_active":"2024-09-11T00:20:37.446029+0000","last_became_peered":"2024-09-11T00:20:37.446029+0000","last_unstale":"2024-09-11T00:22:28.866037+0000","last_undegraded":"2024-09-11T00:22:28.866037+0000","last_fullsized":"2024-09-11T00:22:28.866037+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:58.677406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611496+0000","last_change":"2024-09-11T00:20:27.063086+0000","last_active":"2024-09-11T00:22:56.611496+0000","last_peered":"2024-09-11T00:22:56.611496+0000","last_clean":"2024-09-11T00:22:56.611496+0000","last_became_active":"2024-09-11T00:20:27.062790+0000","last_became_peered":"2024-09-11T00:20:27.062790+0000","last_unstale":"2024-09-11T00:22:56.611496+0000","last_undegraded":"2024-09-11T00:22:56.611496+0000","last_fullsized":"2024-09-11T00:22:56.611496+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:36.657220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679645+0000","last_change":"2024-09-11T00:20:24.535837+0000","last_active":"2024-09-11T00:22:28.679645+0000","last_peered":"2024-09-11T00:22:28.679645+0000","last_clean":"2024-09-11T00:22:28.679645+0000","last_became_active":"2024-09-11T00:20:24.535308+0000","last_became_peered":"2024-09-11T00:20:24.535308+0000","last_unstale":"2024-09-11T00:22:28.679645+0000","last_undegraded":"2024-09-11T00:22:28.679645+0000","last_fullsized":"2024-09-11T00:22:28.679645+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:16.807570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864240+0000","last_change":"2024-09-11T00:20:37.443844+0000","last_active":"2024-09-11T00:22:28.864240+0000","last_peered":"2024-09-11T00:22:28.864240+0000","last_clean":"2024-09-11T00:22:28.864240+0000","last_became_active":"2024-09-11T00:20:37.443374+0000","last_became_peered":"2024-09-11T00:20:37.443374+0000","last_unstale":"2024-09-11T00:22:28.864240+0000","last_undegraded":"2024-09-11T00:22:28.864240+0000","last_fullsized":"2024-09-11T00:22:28.864240+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:47.536788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610053+0000","last_change":"2024-09-11T00:20:27.072874+0000","last_active":"2024-09-11T00:22:56.610053+0000","last_peered":"2024-09-11T00:22:56.610053+0000","last_clean":"2024-09-11T00:22:56.610053+0000","last_became_active":"2024-09-11T00:20:27.072454+0000","last_became_peered":"2024-09-11T00:20:27.072454+0000","last_unstale":"2024-09-11T00:22:56.610053+0000","last_undegraded":"2024-09-11T00:22:56.610053+0000","last_fullsized":"2024-09-11T00:22:56.610053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:25.731037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734678+0000","last_change":"2024-09-11T00:20:24.018929+0000","last_active":"2024-09-11T00:22:28.734678+0000","last_peered":"2024-09-11T00:22:28.734678+0000","last_clean":"2024-09-11T00:22:28.734678+0000","last_became_active":"2024-09-11T00:20:24.018575+0000","last_became_peered":"2024-09-11T00:20:24.018575+0000","last_unstale":"2024-09-11T00:22:28.734678+0000","last_undegraded":"2024-09-11T00:22:28.734678+0000","last_fullsized":"2024-09-11T00:22:28.734678+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:42:45.695561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679083+0000","last_change":"2024-09-11T00:20:37.453468+0000","last_active":"2024-09-11T00:22:28.679083+0000","last_peered":"2024-09-11T00:22:28.679083+0000","last_clean":"2024-09-11T00:22:28.679083+0000","last_became_active":"2024-09-11T00:20:37.453007+0000","last_became_peered":"2024-09-11T00:20:37.453007+0000","last_unstale":"2024-09-11T00:22:28.679083+0000","last_undegraded":"2024-09-11T00:22:28.679083+0000","last_fullsized":"2024-09-11T00:22:28.679083+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:22.979708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604628+0000","last_change":"2024-09-11T00:20:27.064618+0000","last_active":"2024-09-11T00:22:56.604628+0000","last_peered":"2024-09-11T00:22:56.604628+0000","last_clean":"2024-09-11T00:22:56.604628+0000","last_became_active":"2024-09-11T00:20:27.064310+0000","last_became_peered":"2024-09-11T00:20:27.064310+0000","last_unstale":"2024-09-11T00:22:56.604628+0000","last_undegraded":"2024-09-11T00:22:56.604628+0000","last_fullsized":"2024-09-11T00:22:56.604628+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:41.320561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.683037+0000","last_change":"2024-09-11T00:20:24.023522+0000","last_active":"2024-09-11T00:22:28.683037+0000","last_peered":"2024-09-11T00:22:28.683037+0000","last_clean":"2024-09-11T00:22:28.683037+0000","last_became_active":"2024-09-11T00:20:24.022694+0000","last_became_peered":"2024-09-11T00:20:24.022694+0000","last_unstale":"2024-09-11T00:22:28.683037+0000","last_undegraded":"2024-09-11T00:22:28.683037+0000","last_fullsized":"2024-09-11T00:22:28.683037+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:12.374179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733919+0000","last_change":"2024-09-11T00:20:37.441239+0000","last_active":"2024-09-11T00:22:28.733919+0000","last_peered":"2024-09-11T00:22:28.733919+0000","last_clean":"2024-09-11T00:22:28.733919+0000","last_became_active":"2024-09-11T00:20:37.440825+0000","last_became_peered":"2024-09-11T00:20:37.440825+0000","last_unstale":"2024-09-11T00:22:28.733919+0000","last_undegraded":"2024-09-11T00:22:28.733919+0000","last_fullsized":"2024-09-11T00:22:28.733919+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:04.972569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594629+0000","last_change":"2024-09-11T00:20:27.059780+0000","last_active":"2024-09-11T00:22:56.594629+0000","last_peered":"2024-09-11T00:22:56.594629+0000","last_clean":"2024-09-11T00:22:56.594629+0000","last_became_active":"2024-09-11T00:20:27.059411+0000","last_became_peered":"2024-09-11T00:20:27.059411+0000","last_unstale":"2024-09-11T00:22:56.594629+0000","last_undegraded":"2024-09-11T00:22:56.594629+0000","last_fullsized":"2024-09-11T00:22:56.594629+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:22:42.058883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864514+0000","last_change":"2024-09-11T00:20:24.024115+0000","last_active":"2024-09-11T00:22:28.864514+0000","last_peered":"2024-09-11T00:22:28.864514+0000","last_clean":"2024-09-11T00:22:28.864514+0000","last_became_active":"2024-09-11T00:20:24.023729+0000","last_became_peered":"2024-09-11T00:20:24.023729+0000","last_unstale":"2024-09-11T00:22:28.864514+0000","last_undegraded":"2024-09-11T00:22:28.864514+0000","last_fullsized":"2024-09-11T00:22:28.864514+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:15.874168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.733904+0000","last_change":"2024-09-11T00:20:37.443591+0000","last_active":"2024-09-11T00:22:28.733904+0000","last_peered":"2024-09-11T00:22:28.733904+0000","last_clean":"2024-09-11T00:22:28.733904+0000","last_became_active":"2024-09-11T00:20:37.443115+0000","last_became_peered":"2024-09-11T00:20:37.443115+0000","last_unstale":"2024-09-11T00:22:28.733904+0000","last_undegraded":"2024-09-11T00:22:28.733904+0000","last_fullsized":"2024-09-11T00:22:28.733904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:34.310207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607526+0000","last_change":"2024-09-11T00:20:27.061676+0000","last_active":"2024-09-11T00:22:56.607526+0000","last_peered":"2024-09-11T00:22:56.607526+0000","last_clean":"2024-09-11T00:22:56.607526+0000","last_became_active":"2024-09-11T00:20:27.061372+0000","last_became_peered":"2024-09-11T00:20:27.061372+0000","last_unstale":"2024-09-11T00:22:56.607526+0000","last_undegraded":"2024-09-11T00:22:56.607526+0000","last_fullsized":"2024-09-11T00:22:56.607526+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:24.296673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863776+0000","last_change":"2024-09-11T00:21:28.244227+0000","last_active":"2024-09-11T00:22:28.863776+0000","last_peered":"2024-09-11T00:22:28.863776+0000","last_clean":"2024-09-11T00:22:28.863776+0000","last_became_active":"2024-09-11T00:21:28.231547+0000","last_became_peered":"2024-09-11T00:21:28.231547+0000","last_unstale":"2024-09-11T00:22:28.863776+0000","last_undegraded":"2024-09-11T00:22:28.863776+0000","last_fullsized":"2024-09-11T00:22:28.863776+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:35.119308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682992+0000","last_change":"2024-09-11T00:20:37.454603+0000","last_active":"2024-09-11T00:22:28.682992+0000","last_peered":"2024-09-11T00:22:28.682992+0000","last_clean":"2024-09-11T00:22:28.682992+0000","last_became_active":"2024-09-11T00:20:37.454123+0000","last_became_peered":"2024-09-11T00:20:37.454123+0000","last_unstale":"2024-09-11T00:22:28.682992+0000","last_undegraded":"2024-09-11T00:22:28.682992+0000","last_fullsized":"2024-09-11T00:22:28.682992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:09.111768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594320+0000","last_change":"2024-09-11T00:20:27.058806+0000","last_active":"2024-09-11T00:22:56.594320+0000","last_peered":"2024-09-11T00:22:56.594320+0000","last_clean":"2024-09-11T00:22:56.594320+0000","last_became_active":"2024-09-11T00:20:27.058458+0000","last_became_peered":"2024-09-11T00:20:27.058458+0000","last_unstale":"2024-09-11T00:22:56.594320+0000","last_undegraded":"2024-09-11T00:22:56.594320+0000","last_fullsized":"2024-09-11T00:22:56.594320+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:48.168674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'4","reported_seq":182,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:32.408473+0000","last_change":"2024-09-11T00:20:24.022328+0000","last_active":"2024-09-11T00:23:32.408473+0000","last_peered":"2024-09-11T00:23:32.408473+0000","last_clean":"2024-09-11T00:23:32.408473+0000","last_became_active":"2024-09-11T00:20:24.021908+0000","last_became_peered":"2024-09-11T00:20:24.021908+0000","last_unstale":"2024-09-11T00:23:32.408473+0000","last_undegraded":"2024-09-11T00:23:32.408473+0000","last_fullsized":"2024-09-11T00:23:32.408473+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:35.688767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":899,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680798+0000","last_change":"2024-09-11T00:20:37.444898+0000","last_active":"2024-09-11T00:22:28.680798+0000","last_peered":"2024-09-11T00:22:28.680798+0000","last_clean":"2024-09-11T00:22:28.680798+0000","last_became_active":"2024-09-11T00:20:37.443840+0000","last_became_peered":"2024-09-11T00:20:37.443840+0000","last_unstale":"2024-09-11T00:22:28.680798+0000","last_undegraded":"2024-09-11T00:22:28.680798+0000","last_fullsized":"2024-09-11T00:22:28.680798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:58.668882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606050+0000","last_change":"2024-09-11T00:20:27.061198+0000","last_active":"2024-09-11T00:22:56.606050+0000","last_peered":"2024-09-11T00:22:56.606050+0000","last_clean":"2024-09-11T00:22:56.606050+0000","last_became_active":"2024-09-11T00:20:27.060877+0000","last_became_peered":"2024-09-11T00:20:27.060877+0000","last_unstale":"2024-09-11T00:22:56.606050+0000","last_undegraded":"2024-09-11T00:22:56.606050+0000","last_fullsized":"2024-09-11T00:22:56.606050+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:03.595262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734264+0000","last_change":"2024-09-11T00:20:24.022755+0000","last_active":"2024-09-11T00:22:28.734264+0000","last_peered":"2024-09-11T00:22:28.734264+0000","last_clean":"2024-09-11T00:22:28.734264+0000","last_became_active":"2024-09-11T00:20:24.021530+0000","last_became_peered":"2024-09-11T00:20:24.021530+0000","last_unstale":"2024-09-11T00:22:28.734264+0000","last_undegraded":"2024-09-11T00:22:28.734264+0000","last_fullsized":"2024-09-11T00:22:28.734264+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:55:33.881000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864371+0000","last_change":"2024-09-11T00:20:37.443722+0000","last_active":"2024-09-11T00:22:28.864371+0000","last_peered":"2024-09-11T00:22:28.864371+0000","last_clean":"2024-09-11T00:22:28.864371+0000","last_became_active":"2024-09-11T00:20:37.443339+0000","last_became_peered":"2024-09-11T00:20:37.443339+0000","last_unstale":"2024-09-11T00:22:28.864371+0000","last_undegraded":"2024-09-11T00:22:28.864371+0000","last_fullsized":"2024-09-11T00:22:28.864371+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:30.221804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610517+0000","last_change":"2024-09-11T00:20:27.073158+0000","last_active":"2024-09-11T00:22:56.610517+0000","last_peered":"2024-09-11T00:22:56.610517+0000","last_clean":"2024-09-11T00:22:56.610517+0000","last_became_active":"2024-09-11T00:20:27.072805+0000","last_became_peered":"2024-09-11T00:20:27.072805+0000","last_unstale":"2024-09-11T00:22:56.610517+0000","last_undegraded":"2024-09-11T00:22:56.610517+0000","last_fullsized":"2024-09-11T00:22:56.610517+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:35.124632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679703+0000","last_change":"2024-09-11T00:20:24.027178+0000","last_active":"2024-09-11T00:22:28.679703+0000","last_peered":"2024-09-11T00:22:28.679703+0000","last_clean":"2024-09-11T00:22:28.679703+0000","last_became_active":"2024-09-11T00:20:24.026821+0000","last_became_peered":"2024-09-11T00:20:24.026821+0000","last_unstale":"2024-09-11T00:22:28.679703+0000","last_undegraded":"2024-09-11T00:22:28.679703+0000","last_fullsized":"2024-09-11T00:22:28.679703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:38.393135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734422+0000","last_change":"2024-09-11T00:20:37.447587+0000","last_active":"2024-09-11T00:22:28.734422+0000","last_peered":"2024-09-11T00:22:28.734422+0000","last_clean":"2024-09-11T00:22:28.734422+0000","last_became_active":"2024-09-11T00:20:37.447166+0000","last_became_peered":"2024-09-11T00:20:37.447166+0000","last_unstale":"2024-09-11T00:22:28.734422+0000","last_undegraded":"2024-09-11T00:22:28.734422+0000","last_fullsized":"2024-09-11T00:22:28.734422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:05:47.469539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605796+0000","last_change":"2024-09-11T00:20:27.059914+0000","last_active":"2024-09-11T00:22:56.605796+0000","last_peered":"2024-09-11T00:22:56.605796+0000","last_clean":"2024-09-11T00:22:56.605796+0000","last_became_active":"2024-09-11T00:20:27.059595+0000","last_became_peered":"2024-09-11T00:20:27.059595+0000","last_unstale":"2024-09-11T00:22:56.605796+0000","last_undegraded":"2024-09-11T00:22:56.605796+0000","last_fullsized":"2024-09-11T00:22:56.605796+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.360010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'39","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:33.433522+0000","last_change":"2024-09-11T00:20:24.023094+0000","last_active":"2024-09-11T00:23:33.433522+0000","last_peered":"2024-09-11T00:23:33.433522+0000","last_clean":"2024-09-11T00:23:33.433522+0000","last_became_active":"2024-09-11T00:20:24.022742+0000","last_became_peered":"2024-09-11T00:20:24.022742+0000","last_unstale":"2024-09-11T00:23:33.433522+0000","last_undegraded":"2024-09-11T00:23:33.433522+0000","last_fullsized":"2024-09-11T00:23:33.433522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:24:41.354435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68005,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":270,"num_read_kb":202,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.680599+0000","last_change":"2024-09-11T00:20:37.451610+0000","last_active":"2024-09-11T00:22:28.680599+0000","last_peered":"2024-09-11T00:22:28.680599+0000","last_clean":"2024-09-11T00:22:28.680599+0000","last_became_active":"2024-09-11T00:20:37.444233+0000","last_became_peered":"2024-09-11T00:20:37.444233+0000","last_unstale":"2024-09-11T00:22:28.680599+0000","last_undegraded":"2024-09-11T00:22:28.680599+0000","last_fullsized":"2024-09-11T00:22:28.680599+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:02.558457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608959+0000","last_change":"2024-09-11T00:20:27.061229+0000","last_active":"2024-09-11T00:22:56.608959+0000","last_peered":"2024-09-11T00:22:56.608959+0000","last_clean":"2024-09-11T00:22:56.608959+0000","last_became_active":"2024-09-11T00:20:27.060360+0000","last_became_peered":"2024-09-11T00:20:27.060360+0000","last_unstale":"2024-09-11T00:22:56.608959+0000","last_undegraded":"2024-09-11T00:22:56.608959+0000","last_fullsized":"2024-09-11T00:22:56.608959+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:40:43.107405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734591+0000","last_change":"2024-09-11T00:20:24.036483+0000","last_active":"2024-09-11T00:22:28.734591+0000","last_peered":"2024-09-11T00:22:28.734591+0000","last_clean":"2024-09-11T00:22:28.734591+0000","last_became_active":"2024-09-11T00:20:24.035855+0000","last_became_peered":"2024-09-11T00:20:24.035855+0000","last_unstale":"2024-09-11T00:22:28.734591+0000","last_undegraded":"2024-09-11T00:22:28.734591+0000","last_fullsized":"2024-09-11T00:22:28.734591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.611169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864762+0000","last_change":"2024-09-11T00:21:28.245665+0000","last_active":"2024-09-11T00:22:28.864762+0000","last_peered":"2024-09-11T00:22:28.864762+0000","last_clean":"2024-09-11T00:22:28.864762+0000","last_became_active":"2024-09-11T00:21:28.245221+0000","last_became_peered":"2024-09-11T00:21:28.245221+0000","last_unstale":"2024-09-11T00:22:28.864762+0000","last_undegraded":"2024-09-11T00:22:28.864762+0000","last_fullsized":"2024-09-11T00:22:28.864762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:36.627363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611454+0000","last_change":"2024-09-11T00:21:28.231435+0000","last_active":"2024-09-11T00:22:56.611454+0000","last_peered":"2024-09-11T00:22:56.611454+0000","last_clean":"2024-09-11T00:22:56.611454+0000","last_became_active":"2024-09-11T00:21:28.230956+0000","last_became_peered":"2024-09-11T00:21:28.230956+0000","last_unstale":"2024-09-11T00:22:56.611454+0000","last_undegraded":"2024-09-11T00:22:56.611454+0000","last_fullsized":"2024-09-11T00:22:56.611454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:28.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.865542+0000","last_change":"2024-09-11T00:20:24.023068+0000","last_active":"2024-09-11T00:22:28.865542+0000","last_peered":"2024-09-11T00:22:28.865542+0000","last_clean":"2024-09-11T00:22:28.865542+0000","last_became_active":"2024-09-11T00:20:24.022718+0000","last_became_peered":"2024-09-11T00:20:24.022718+0000","last_unstale":"2024-09-11T00:22:28.865542+0000","last_undegraded":"2024-09-11T00:22:28.865542+0000","last_fullsized":"2024-09-11T00:22:28.865542+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:23.651536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863633+0000","last_change":"2024-09-11T00:21:28.231887+0000","last_active":"2024-09-11T00:22:28.863633+0000","last_peered":"2024-09-11T00:22:28.863633+0000","last_clean":"2024-09-11T00:22:28.863633+0000","last_became_active":"2024-09-11T00:21:28.231456+0000","last_became_peered":"2024-09-11T00:21:28.231456+0000","last_unstale":"2024-09-11T00:22:28.863633+0000","last_undegraded":"2024-09-11T00:22:28.863633+0000","last_fullsized":"2024-09-11T00:22:28.863633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:54:32.712598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613303+0000","last_change":"2024-09-11T00:20:27.065109+0000","last_active":"2024-09-11T00:22:56.613303+0000","last_peered":"2024-09-11T00:22:56.613303+0000","last_clean":"2024-09-11T00:22:56.613303+0000","last_became_active":"2024-09-11T00:20:27.064677+0000","last_became_peered":"2024-09-11T00:20:27.064677+0000","last_unstale":"2024-09-11T00:22:56.613303+0000","last_undegraded":"2024-09-11T00:22:56.613303+0000","last_fullsized":"2024-09-11T00:22:56.613303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:05:19.430026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":179,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:32.407604+0000","last_change":"2024-09-11T00:20:24.025347+0000","last_active":"2024-09-11T00:23:32.407604+0000","last_peered":"2024-09-11T00:23:32.407604+0000","last_clean":"2024-09-11T00:23:32.407604+0000","last_became_active":"2024-09-11T00:20:24.024980+0000","last_became_peered":"2024-09-11T00:20:24.024980+0000","last_unstale":"2024-09-11T00:23:32.407604+0000","last_undegraded":"2024-09-11T00:23:32.407604+0000","last_fullsized":"2024-09-11T00:23:32.407604+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:12.530339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.863597+0000","last_change":"2024-09-11T00:20:37.440340+0000","last_active":"2024-09-11T00:22:28.863597+0000","last_peered":"2024-09-11T00:22:28.863597+0000","last_clean":"2024-09-11T00:22:28.863597+0000","last_became_active":"2024-09-11T00:20:37.439896+0000","last_became_peered":"2024-09-11T00:20:37.439896+0000","last_unstale":"2024-09-11T00:22:28.863597+0000","last_undegraded":"2024-09-11T00:22:28.863597+0000","last_fullsized":"2024-09-11T00:22:28.863597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:38.774065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679198+0000","last_change":"2024-09-11T00:20:37.445860+0000","last_active":"2024-09-11T00:22:28.679198+0000","last_peered":"2024-09-11T00:22:28.679198+0000","last_clean":"2024-09-11T00:22:28.679198+0000","last_became_active":"2024-09-11T00:20:37.445450+0000","last_became_peered":"2024-09-11T00:20:37.445450+0000","last_unstale":"2024-09-11T00:22:28.679198+0000","last_undegraded":"2024-09-11T00:22:28.679198+0000","last_fullsized":"2024-09-11T00:22:28.679198+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:44.512158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607656+0000","last_change":"2024-09-11T00:20:27.073529+0000","last_active":"2024-09-11T00:22:56.607656+0000","last_peered":"2024-09-11T00:22:56.607656+0000","last_clean":"2024-09-11T00:22:56.607656+0000","last_became_active":"2024-09-11T00:20:27.073168+0000","last_became_peered":"2024-09-11T00:20:27.073168+0000","last_unstale":"2024-09-11T00:22:56.607656+0000","last_undegraded":"2024-09-11T00:22:56.607656+0000","last_fullsized":"2024-09-11T00:22:56.607656+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:17:16.407543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.734124+0000","last_change":"2024-09-11T00:20:24.021778+0000","last_active":"2024-09-11T00:22:28.734124+0000","last_peered":"2024-09-11T00:22:28.734124+0000","last_clean":"2024-09-11T00:22:28.734124+0000","last_became_active":"2024-09-11T00:20:24.021443+0000","last_became_peered":"2024-09-11T00:20:24.021443+0000","last_unstale":"2024-09-11T00:22:28.734124+0000","last_undegraded":"2024-09-11T00:22:28.734124+0000","last_fullsized":"2024-09-11T00:22:28.734124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.864397+0000","last_change":"2024-09-11T00:20:37.444279+0000","last_active":"2024-09-11T00:22:28.864397+0000","last_peered":"2024-09-11T00:22:28.864397+0000","last_clean":"2024-09-11T00:22:28.864397+0000","last_became_active":"2024-09-11T00:20:37.443821+0000","last_became_peered":"2024-09-11T00:20:37.443821+0000","last_unstale":"2024-09-11T00:22:28.864397+0000","last_undegraded":"2024-09-11T00:22:28.864397+0000","last_fullsized":"2024-09-11T00:22:28.864397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:10.257926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606265+0000","last_change":"2024-09-11T00:20:27.066155+0000","last_active":"2024-09-11T00:22:56.606265+0000","last_peered":"2024-09-11T00:22:56.606265+0000","last_clean":"2024-09-11T00:22:56.606265+0000","last_became_active":"2024-09-11T00:20:27.057262+0000","last_became_peered":"2024-09-11T00:20:27.057262+0000","last_unstale":"2024-09-11T00:22:56.606265+0000","last_undegraded":"2024-09-11T00:22:56.606265+0000","last_fullsized":"2024-09-11T00:22:56.606265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:54.160215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.682647+0000","last_change":"2024-09-11T00:20:24.026123+0000","last_active":"2024-09-11T00:22:28.682647+0000","last_peered":"2024-09-11T00:22:28.682647+0000","last_clean":"2024-09-11T00:22:28.682647+0000","last_became_active":"2024-09-11T00:20:24.025832+0000","last_became_peered":"2024-09-11T00:20:24.025832+0000","last_unstale":"2024-09-11T00:22:28.682647+0000","last_undegraded":"2024-09-11T00:22:28.682647+0000","last_fullsized":"2024-09-11T00:22:28.682647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:50:45.444840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.830187+0000","last_change":"2024-09-11T00:20:24.028523+0000","last_active":"2024-09-11T00:22:28.830187+0000","last_peered":"2024-09-11T00:22:28.830187+0000","last_clean":"2024-09-11T00:22:28.830187+0000","last_became_active":"2024-09-11T00:20:24.027801+0000","last_became_peered":"2024-09-11T00:20:24.027801+0000","last_unstale":"2024-09-11T00:22:28.830187+0000","last_undegraded":"2024-09-11T00:22:28.830187+0000","last_fullsized":"2024-09-11T00:22:28.830187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:14.174983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607124+0000","last_change":"2024-09-11T00:20:27.074634+0000","last_active":"2024-09-11T00:22:56.607124+0000","last_peered":"2024-09-11T00:22:56.607124+0000","last_clean":"2024-09-11T00:22:56.607124+0000","last_became_active":"2024-09-11T00:20:27.074292+0000","last_became_peered":"2024-09-11T00:20:27.074292+0000","last_unstale":"2024-09-11T00:22:56.607124+0000","last_undegraded":"2024-09-11T00:22:56.607124+0000","last_fullsized":"2024-09-11T00:22:56.607124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:43:07.467262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679446+0000","last_change":"2024-09-11T00:20:37.454341+0000","last_active":"2024-09-11T00:22:28.679446+0000","last_peered":"2024-09-11T00:22:28.679446+0000","last_clean":"2024-09-11T00:22:28.679446+0000","last_became_active":"2024-09-11T00:20:37.453868+0000","last_became_peered":"2024-09-11T00:20:37.453868+0000","last_unstale":"2024-09-11T00:22:28.679446+0000","last_undegraded":"2024-09-11T00:22:28.679446+0000","last_fullsized":"2024-09-11T00:22:28.679446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:57:06.015338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'7","reported_seq":186,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:23:32.408380+0000","last_change":"2024-09-11T00:20:24.025184+0000","last_active":"2024-09-11T00:23:32.408380+0000","last_peered":"2024-09-11T00:23:32.408380+0000","last_clean":"2024-09-11T00:23:32.408380+0000","last_became_active":"2024-09-11T00:20:24.024670+0000","last_became_peered":"2024-09-11T00:20:24.024670+0000","last_unstale":"2024-09-11T00:23:32.408380+0000","last_undegraded":"2024-09-11T00:23:32.408380+0000","last_fullsized":"2024-09-11T00:23:32.408380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:22.999977+0000","last_clean_scrub_stamp":"2024-09-11T00:20:22.999977+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:40.579280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.594092+0000","last_change":"2024-09-11T00:20:27.057822+0000","last_active":"2024-09-11T00:22:56.594092+0000","last_peered":"2024-09-11T00:22:56.594092+0000","last_clean":"2024-09-11T00:22:56.594092+0000","last_became_active":"2024-09-11T00:20:27.057518+0000","last_became_peered":"2024-09-11T00:20:27.057518+0000","last_unstale":"2024-09-11T00:22:56.594092+0000","last_undegraded":"2024-09-11T00:22:56.594092+0000","last_fullsized":"2024-09-11T00:22:56.594092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:00.264327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:28.679539+0000","last_change":"2024-09-11T00:20:37.438369+0000","last_active":"2024-09-11T00:22:28.679539+0000","last_peered":"2024-09-11T00:22:28.679539+0000","last_clean":"2024-09-11T00:22:28.679539+0000","last_became_active":"2024-09-11T00:20:37.437942+0000","last_became_peered":"2024-09-11T00:20:37.437942+0000","last_unstale":"2024-09-11T00:22:28.679539+0000","last_undegraded":"2024-09-11T00:22:28.679539+0000","last_fullsized":"2024-09-11T00:22:28.679539+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:36.412555+0000","last_clean_scrub_stamp":"2024-09-11T00:20:36.412555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:13.410905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.595081+0000","last_change":"2024-09-11T00:20:27.057449+0000","last_active":"2024-09-11T00:22:56.595081+0000","last_peered":"2024-09-11T00:22:56.595081+0000","last_clean":"2024-09-11T00:22:56.595081+0000","last_became_active":"2024-09-11T00:20:27.057146+0000","last_became_peered":"2024-09-11T00:20:27.057146+0000","last_unstale":"2024-09-11T00:22:56.595081+0000","last_undegraded":"2024-09-11T00:22:56.595081+0000","last_fullsized":"2024-09-11T00:22:56.595081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:57.895062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.603116+0000","last_change":"2024-09-11T00:20:27.061515+0000","last_active":"2024-09-11T00:22:56.603116+0000","last_peered":"2024-09-11T00:22:56.603116+0000","last_clean":"2024-09-11T00:22:56.603116+0000","last_became_active":"2024-09-11T00:20:27.061175+0000","last_became_peered":"2024-09-11T00:20:27.061175+0000","last_unstale":"2024-09-11T00:22:56.603116+0000","last_undegraded":"2024-09-11T00:22:56.603116+0000","last_fullsized":"2024-09-11T00:22:56.603116+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:03.112574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.609556+0000","last_change":"2024-09-11T00:20:27.063015+0000","last_active":"2024-09-11T00:22:56.609556+0000","last_peered":"2024-09-11T00:22:56.609556+0000","last_clean":"2024-09-11T00:22:56.609556+0000","last_became_active":"2024-09-11T00:20:27.062689+0000","last_became_peered":"2024-09-11T00:20:27.062689+0000","last_unstale":"2024-09-11T00:22:56.609556+0000","last_undegraded":"2024-09-11T00:22:56.609556+0000","last_fullsized":"2024-09-11T00:22:56.609556+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:23.736073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605982+0000","last_change":"2024-09-11T00:20:27.062196+0000","last_active":"2024-09-11T00:22:56.605982+0000","last_peered":"2024-09-11T00:22:56.605982+0000","last_clean":"2024-09-11T00:22:56.605982+0000","last_became_active":"2024-09-11T00:20:27.061918+0000","last_became_peered":"2024-09-11T00:20:27.061918+0000","last_unstale":"2024-09-11T00:22:56.605982+0000","last_undegraded":"2024-09-11T00:22:56.605982+0000","last_fullsized":"2024-09-11T00:22:56.605982+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:45.799616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611720+0000","last_change":"2024-09-11T00:20:27.059021+0000","last_active":"2024-09-11T00:22:56.611720+0000","last_peered":"2024-09-11T00:22:56.611720+0000","last_clean":"2024-09-11T00:22:56.611720+0000","last_became_active":"2024-09-11T00:20:27.058726+0000","last_became_peered":"2024-09-11T00:20:27.058726+0000","last_unstale":"2024-09-11T00:22:56.611720+0000","last_undegraded":"2024-09-11T00:22:56.611720+0000","last_fullsized":"2024-09-11T00:22:56.611720+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:23.900236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.593828+0000","last_change":"2024-09-11T00:20:27.057593+0000","last_active":"2024-09-11T00:22:56.593828+0000","last_peered":"2024-09-11T00:22:56.593828+0000","last_clean":"2024-09-11T00:22:56.593828+0000","last_became_active":"2024-09-11T00:20:27.057310+0000","last_became_peered":"2024-09-11T00:20:27.057310+0000","last_unstale":"2024-09-11T00:22:56.593828+0000","last_undegraded":"2024-09-11T00:22:56.593828+0000","last_fullsized":"2024-09-11T00:22:56.593828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:52.355218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605460+0000","last_change":"2024-09-11T00:20:27.061140+0000","last_active":"2024-09-11T00:22:56.605460+0000","last_peered":"2024-09-11T00:22:56.605460+0000","last_clean":"2024-09-11T00:22:56.605460+0000","last_became_active":"2024-09-11T00:20:27.060500+0000","last_became_peered":"2024-09-11T00:20:27.060500+0000","last_unstale":"2024-09-11T00:22:56.605460+0000","last_undegraded":"2024-09-11T00:22:56.605460+0000","last_fullsized":"2024-09-11T00:22:56.605460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.159068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.591859+0000","last_change":"2024-09-11T00:20:27.061936+0000","last_active":"2024-09-11T00:22:56.591859+0000","last_peered":"2024-09-11T00:22:56.591859+0000","last_clean":"2024-09-11T00:22:56.591859+0000","last_became_active":"2024-09-11T00:20:27.061616+0000","last_became_peered":"2024-09-11T00:20:27.061616+0000","last_unstale":"2024-09-11T00:22:56.591859+0000","last_undegraded":"2024-09-11T00:22:56.591859+0000","last_fullsized":"2024-09-11T00:22:56.591859+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:32.185634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.608892+0000","last_change":"2024-09-11T00:20:27.074645+0000","last_active":"2024-09-11T00:22:56.608892+0000","last_peered":"2024-09-11T00:22:56.608892+0000","last_clean":"2024-09-11T00:22:56.608892+0000","last_became_active":"2024-09-11T00:20:27.074355+0000","last_became_peered":"2024-09-11T00:20:27.074355+0000","last_unstale":"2024-09-11T00:22:56.608892+0000","last_undegraded":"2024-09-11T00:22:56.608892+0000","last_fullsized":"2024-09-11T00:22:56.608892+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:02.361668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.601812+0000","last_change":"2024-09-11T00:20:27.069588+0000","last_active":"2024-09-11T00:22:56.601812+0000","last_peered":"2024-09-11T00:22:56.601812+0000","last_clean":"2024-09-11T00:22:56.601812+0000","last_became_active":"2024-09-11T00:20:27.067403+0000","last_became_peered":"2024-09-11T00:20:27.067403+0000","last_unstale":"2024-09-11T00:22:56.601812+0000","last_undegraded":"2024-09-11T00:22:56.601812+0000","last_fullsized":"2024-09-11T00:22:56.601812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:29.439013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610162+0000","last_change":"2024-09-11T00:20:27.071586+0000","last_active":"2024-09-11T00:22:56.610162+0000","last_peered":"2024-09-11T00:22:56.610162+0000","last_clean":"2024-09-11T00:22:56.610162+0000","last_became_active":"2024-09-11T00:20:27.071072+0000","last_became_peered":"2024-09-11T00:20:27.071072+0000","last_unstale":"2024-09-11T00:22:56.610162+0000","last_undegraded":"2024-09-11T00:22:56.610162+0000","last_fullsized":"2024-09-11T00:22:56.610162+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:43:34.133027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613642+0000","last_change":"2024-09-11T00:20:27.066683+0000","last_active":"2024-09-11T00:22:56.613642+0000","last_peered":"2024-09-11T00:22:56.613642+0000","last_clean":"2024-09-11T00:22:56.613642+0000","last_became_active":"2024-09-11T00:20:27.066355+0000","last_became_peered":"2024-09-11T00:20:27.066355+0000","last_unstale":"2024-09-11T00:22:56.613642+0000","last_undegraded":"2024-09-11T00:22:56.613642+0000","last_fullsized":"2024-09-11T00:22:56.613642+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:26.868072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.598228+0000","last_change":"2024-09-11T00:20:27.060546+0000","last_active":"2024-09-11T00:22:56.598228+0000","last_peered":"2024-09-11T00:22:56.598228+0000","last_clean":"2024-09-11T00:22:56.598228+0000","last_became_active":"2024-09-11T00:20:27.059619+0000","last_became_peered":"2024-09-11T00:20:27.059619+0000","last_unstale":"2024-09-11T00:22:56.598228+0000","last_undegraded":"2024-09-11T00:22:56.598228+0000","last_fullsized":"2024-09-11T00:22:56.598228+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:41.319854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.607755+0000","last_change":"2024-09-11T00:20:27.073342+0000","last_active":"2024-09-11T00:22:56.607755+0000","last_peered":"2024-09-11T00:22:56.607755+0000","last_clean":"2024-09-11T00:22:56.607755+0000","last_became_active":"2024-09-11T00:20:27.072435+0000","last_became_peered":"2024-09-11T00:20:27.072435+0000","last_unstale":"2024-09-11T00:22:56.607755+0000","last_undegraded":"2024-09-11T00:22:56.607755+0000","last_fullsized":"2024-09-11T00:22:56.607755+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:15.943515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612461+0000","last_change":"2024-09-11T00:20:27.063220+0000","last_active":"2024-09-11T00:22:56.612461+0000","last_peered":"2024-09-11T00:22:56.612461+0000","last_clean":"2024-09-11T00:22:56.612461+0000","last_became_active":"2024-09-11T00:20:27.062823+0000","last_became_peered":"2024-09-11T00:20:27.062823+0000","last_unstale":"2024-09-11T00:22:56.612461+0000","last_undegraded":"2024-09-11T00:22:56.612461+0000","last_fullsized":"2024-09-11T00:22:56.612461+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:22.430295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.613071+0000","last_change":"2024-09-11T00:20:27.062088+0000","last_active":"2024-09-11T00:22:56.613071+0000","last_peered":"2024-09-11T00:22:56.613071+0000","last_clean":"2024-09-11T00:22:56.613071+0000","last_became_active":"2024-09-11T00:20:27.061639+0000","last_became_peered":"2024-09-11T00:20:27.061639+0000","last_unstale":"2024-09-11T00:22:56.613071+0000","last_undegraded":"2024-09-11T00:22:56.613071+0000","last_fullsized":"2024-09-11T00:22:56.613071+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.965908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.611123+0000","last_change":"2024-09-11T00:20:27.067170+0000","last_active":"2024-09-11T00:22:56.611123+0000","last_peered":"2024-09-11T00:22:56.611123+0000","last_clean":"2024-09-11T00:22:56.611123+0000","last_became_active":"2024-09-11T00:20:27.066851+0000","last_became_peered":"2024-09-11T00:20:27.066851+0000","last_unstale":"2024-09-11T00:22:56.611123+0000","last_undegraded":"2024-09-11T00:22:56.611123+0000","last_fullsized":"2024-09-11T00:22:56.611123+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:51.312605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604750+0000","last_change":"2024-09-11T00:21:28.233390+0000","last_active":"2024-09-11T00:22:56.604750+0000","last_peered":"2024-09-11T00:22:56.604750+0000","last_clean":"2024-09-11T00:22:56.604750+0000","last_became_active":"2024-09-11T00:21:28.232886+0000","last_became_peered":"2024-09-11T00:21:28.232886+0000","last_unstale":"2024-09-11T00:22:56.604750+0000","last_undegraded":"2024-09-11T00:22:56.604750+0000","last_fullsized":"2024-09-11T00:22:56.604750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:18:26.702069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606581+0000","last_change":"2024-09-11T00:20:27.059553+0000","last_active":"2024-09-11T00:22:56.606581+0000","last_peered":"2024-09-11T00:22:56.606581+0000","last_clean":"2024-09-11T00:22:56.606581+0000","last_became_active":"2024-09-11T00:20:27.058979+0000","last_became_peered":"2024-09-11T00:20:27.058979+0000","last_unstale":"2024-09-11T00:22:56.606581+0000","last_undegraded":"2024-09-11T00:22:56.606581+0000","last_fullsized":"2024-09-11T00:22:56.606581+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:21.472828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.606329+0000","last_change":"2024-09-11T00:22:28.831536+0000","last_active":"2024-09-11T00:22:56.606329+0000","last_peered":"2024-09-11T00:22:56.606329+0000","last_clean":"2024-09-11T00:22:56.606329+0000","last_became_active":"2024-09-11T00:22:28.831151+0000","last_became_peered":"2024-09-11T00:22:28.831151+0000","last_unstale":"2024-09-11T00:22:56.606329+0000","last_undegraded":"2024-09-11T00:22:56.606329+0000","last_fullsized":"2024-09-11T00:22:56.606329+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:17.532570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.602775+0000","last_change":"2024-09-11T00:20:27.059140+0000","last_active":"2024-09-11T00:22:56.602775+0000","last_peered":"2024-09-11T00:22:56.602775+0000","last_clean":"2024-09-11T00:22:56.602775+0000","last_became_active":"2024-09-11T00:20:27.058765+0000","last_became_peered":"2024-09-11T00:20:27.058765+0000","last_unstale":"2024-09-11T00:22:56.602775+0000","last_undegraded":"2024-09-11T00:22:56.602775+0000","last_fullsized":"2024-09-11T00:22:56.602775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:43.947411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.604652+0000","last_change":"2024-09-11T00:20:27.061149+0000","last_active":"2024-09-11T00:22:56.604652+0000","last_peered":"2024-09-11T00:22:56.604652+0000","last_clean":"2024-09-11T00:22:56.604652+0000","last_became_active":"2024-09-11T00:20:27.059949+0000","last_became_peered":"2024-09-11T00:20:27.059949+0000","last_unstale":"2024-09-11T00:22:56.604652+0000","last_undegraded":"2024-09-11T00:22:56.604652+0000","last_fullsized":"2024-09-11T00:22:56.604652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:16:03.401942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597666+0000","last_change":"2024-09-11T00:21:28.231322+0000","last_active":"2024-09-11T00:22:56.597666+0000","last_peered":"2024-09-11T00:22:56.597666+0000","last_clean":"2024-09-11T00:22:56.597666+0000","last_became_active":"2024-09-11T00:21:28.231075+0000","last_became_peered":"2024-09-11T00:21:28.231075+0000","last_unstale":"2024-09-11T00:22:56.597666+0000","last_undegraded":"2024-09-11T00:22:56.597666+0000","last_fullsized":"2024-09-11T00:22:56.597666+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:50.887001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612406+0000","last_change":"2024-09-11T00:20:27.058175+0000","last_active":"2024-09-11T00:22:56.612406+0000","last_peered":"2024-09-11T00:22:56.612406+0000","last_clean":"2024-09-11T00:22:56.612406+0000","last_became_active":"2024-09-11T00:20:27.057797+0000","last_became_peered":"2024-09-11T00:20:27.057797+0000","last_unstale":"2024-09-11T00:22:56.612406+0000","last_undegraded":"2024-09-11T00:22:56.612406+0000","last_fullsized":"2024-09-11T00:22:56.612406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:29.370400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.609823+0000","last_change":"2024-09-11T00:20:27.062024+0000","last_active":"2024-09-11T00:22:56.609823+0000","last_peered":"2024-09-11T00:22:56.609823+0000","last_clean":"2024-09-11T00:22:56.609823+0000","last_became_active":"2024-09-11T00:20:27.061763+0000","last_became_peered":"2024-09-11T00:20:27.061763+0000","last_unstale":"2024-09-11T00:22:56.609823+0000","last_undegraded":"2024-09-11T00:22:56.609823+0000","last_fullsized":"2024-09-11T00:22:56.609823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:07.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.597966+0000","last_change":"2024-09-11T00:20:27.060258+0000","last_active":"2024-09-11T00:22:56.597966+0000","last_peered":"2024-09-11T00:22:56.597966+0000","last_clean":"2024-09-11T00:22:56.597966+0000","last_became_active":"2024-09-11T00:20:27.059475+0000","last_became_peered":"2024-09-11T00:20:27.059475+0000","last_unstale":"2024-09-11T00:22:56.597966+0000","last_undegraded":"2024-09-11T00:22:56.597966+0000","last_fullsized":"2024-09-11T00:22:56.597966+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:27.186322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.603092+0000","last_change":"2024-09-11T00:20:27.059730+0000","last_active":"2024-09-11T00:22:56.603092+0000","last_peered":"2024-09-11T00:22:56.603092+0000","last_clean":"2024-09-11T00:22:56.603092+0000","last_became_active":"2024-09-11T00:20:27.059389+0000","last_became_peered":"2024-09-11T00:20:27.059389+0000","last_unstale":"2024-09-11T00:22:56.603092+0000","last_undegraded":"2024-09-11T00:22:56.603092+0000","last_fullsized":"2024-09-11T00:22:56.603092+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.445159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605141+0000","last_change":"2024-09-11T00:20:27.072765+0000","last_active":"2024-09-11T00:22:56.605141+0000","last_peered":"2024-09-11T00:22:56.605141+0000","last_clean":"2024-09-11T00:22:56.605141+0000","last_became_active":"2024-09-11T00:20:27.072392+0000","last_became_peered":"2024-09-11T00:20:27.072392+0000","last_unstale":"2024-09-11T00:22:56.605141+0000","last_undegraded":"2024-09-11T00:22:56.605141+0000","last_fullsized":"2024-09-11T00:22:56.605141+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:27.030976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.600287+0000","last_change":"2024-09-11T00:20:27.056575+0000","last_active":"2024-09-11T00:22:56.600287+0000","last_peered":"2024-09-11T00:22:56.600287+0000","last_clean":"2024-09-11T00:22:56.600287+0000","last_became_active":"2024-09-11T00:20:27.056264+0000","last_became_peered":"2024-09-11T00:20:27.056264+0000","last_unstale":"2024-09-11T00:22:56.600287+0000","last_undegraded":"2024-09-11T00:22:56.600287+0000","last_fullsized":"2024-09-11T00:22:56.600287+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:39.722518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.610079+0000","last_change":"2024-09-11T00:20:27.065153+0000","last_active":"2024-09-11T00:22:56.610079+0000","last_peered":"2024-09-11T00:22:56.610079+0000","last_clean":"2024-09-11T00:22:56.610079+0000","last_became_active":"2024-09-11T00:20:27.062978+0000","last_became_peered":"2024-09-11T00:20:27.062978+0000","last_unstale":"2024-09-11T00:22:56.610079+0000","last_undegraded":"2024-09-11T00:22:56.610079+0000","last_fullsized":"2024-09-11T00:22:56.610079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:37.820467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.605823+0000","last_change":"2024-09-11T00:20:27.070085+0000","last_active":"2024-09-11T00:22:56.605823+0000","last_peered":"2024-09-11T00:22:56.605823+0000","last_clean":"2024-09-11T00:22:56.605823+0000","last_became_active":"2024-09-11T00:20:27.069809+0000","last_became_peered":"2024-09-11T00:20:27.069809+0000","last_unstale":"2024-09-11T00:22:56.605823+0000","last_undegraded":"2024-09-11T00:22:56.605823+0000","last_fullsized":"2024-09-11T00:22:56.605823+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:01.048590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"89'8","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-11T00:22:56.612973+0000","last_change":"2024-09-11T00:20:27.064584+0000","last_active":"2024-09-11T00:22:56.612973+0000","last_peered":"2024-09-11T00:22:56.612973+0000","last_clean":"2024-09-11T00:22:56.612973+0000","last_became_active":"2024-09-11T00:20:27.064271+0000","last_became_peered":"2024-09-11T00:20:27.064271+0000","last_unstale":"2024-09-11T00:22:56.612973+0000","last_undegraded":"2024-09-11T00:22:56.612973+0000","last_fullsized":"2024-09-11T00:22:56.612973+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T00:20:26.025630+0000","last_clean_scrub_stamp":"2024-09-11T00:20:26.025630+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:53.717744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72458,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":803,"num_read_kb":477,"num_write":183,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":217374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710771,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":515118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907000,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28704,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716448,"statfs":{"total":95995035648,"available":95965642752,"internally_reserved":0,"allocated":1720320,"data_stored":1170590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070526,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":519338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266756,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":515050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462985,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32780,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712372,"statfs":{"total":95995035648,"available":95961468928,"internally_reserved":0,"allocated":1634304,"data_stored":1111523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1587,"internal_metadata":31914445},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626510,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":510418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822741,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1642496,"data_stored":1106800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986267,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":515379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182496,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":510612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1118208,"data_stored":580215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":578333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738480,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":511806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T00:23:36.077 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:23:36.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:36 smithi103 ceph-mon[30578]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:36.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:36 smithi117 ceph-mon[35525]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:36.451 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:36.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:36 smithi183 ceph-mon[35934]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T00:23:37.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 -- 172.21.15.103:0/2433768614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 msgr2=0x7f3fec108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/2433768614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec108a60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009900 tx=0x7f3fd402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.082 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 -- 172.21.15.103:0/2433768614 shutdown_connections 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/2433768614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fec108fa0 0x7f3fec10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/2433768614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec108a60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/2433768614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fec102c10 0x7f3fec102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 -- 172.21.15.103:0/2433768614 >> 172.21.15.103:0/2433768614 conn(0x7f3fec0fe660 msgr2=0x7f3fec100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 -- 172.21.15.103:0/2433768614 shutdown_connections 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.080+0000 7f3ff0c00640 1 -- 172.21.15.103:0/2433768614 wait complete. 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.081+0000 7f3ff0c00640 1 Processor -- start 2024-09-11T00:23:37.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3ff0c00640 1 -- start start 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3ff0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fec102c10 0x7f3fec19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3ff0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3ff0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fec108fa0 0x7f3fec1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3ff0c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec112c70 con 0x7f3fec108620 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3ff0c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec112af0 con 0x7f3fec108fa0 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3ff0c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec112df0 con 0x7f3fec102c10 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3fe3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33466/0 (socket says 172.21.15.103:33466) 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3feb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fec102c10 0x7f3fec19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3fe3e00640 1 -- 172.21.15.103:0/3045069358 learned_addr learned my addr 172.21.15.103:0/3045069358 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:37.085 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.082+0000 7f3febe00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fec108fa0 0x7f3fec1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3feb400640 1 -- 172.21.15.103:0/3045069358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fec108fa0 msgr2=0x7f3fec1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3feb400640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fec108fa0 0x7f3fec1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3feb400640 1 -- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 msgr2=0x7f3fec19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3feb400640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3feb400640 1 -- 172.21.15.103:0/3045069358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec1a1350 con 0x7f3fec102c10 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3fe3e00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3feb400640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fec102c10 0x7f3fec19b2a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9f0 tx=0x7f3fdc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3fe8c00640 1 -- 172.21.15.103:0/3045069358 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00ccc0 con 0x7f3fec102c10 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec1a1640 con 0x7f3fec102c10 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3fe8c00640 1 -- 172.21.15.103:0/3045069358 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fdc0108e0 con 0x7f3fec102c10 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.083+0000 7f3fe8c00640 1 -- 172.21.15.103:0/3045069358 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc018880 con 0x7f3fec102c10 2024-09-11T00:23:37.086 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.084+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec1a7d00 con 0x7f3fec102c10 2024-09-11T00:23:37.087 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.084+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fec102ff0 con 0x7f3fec102c10 2024-09-11T00:23:37.088 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.087+0000 7f3fe8c00640 1 -- 172.21.15.103:0/3045069358 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3fdc010430 con 0x7f3fec102c10 2024-09-11T00:23:37.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.088+0000 7f3fe8c00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3fd00b5390 0x7f3fd00b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:37.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.089+0000 7f3fe8c00640 1 -- 172.21.15.103:0/3045069358 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3fdc0e6740 con 0x7f3fec102c10 2024-09-11T00:23:37.089 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.089+0000 7f3fe3e00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3fd00b5390 0x7f3fd00b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:37.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.089+0000 7f3fe3e00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3fd00b5390 0x7f3fd00b7850 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3fec104e10 tx=0x7f3fd4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:37.090 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.090+0000 7f3fe8c00640 1 -- 172.21.15.103:0/3045069358 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3fdc0acbc0 con 0x7f3fec102c10 2024-09-11T00:23:37.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.315+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fec19c3d0 con 0x7f3fec102c10 2024-09-11T00:23:37.317 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:37 smithi103 ceph-mon[30578]: pgmap v414: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T00:23:37.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.316+0000 7f3fe8c00640 1 -- 172.21.15.103:0/3045069358 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11933 (secure 0 0 0) 0x7f3fdc0b15f0 con 0x7f3fec102c10 2024-09-11T00:23:37.321 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:37.321 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-11T00:23:29:477265+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:23:28.708651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:23:37.322 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-11T00:23:37.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3fd00b5390 msgr2=0x7f3fd00b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3fd00b5390 0x7f3fd00b7850 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3fec104e10 tx=0x7f3fd4005a20 comp rx=0 tx=0).stop 2024-09-11T00:23:37.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fec102c10 msgr2=0x7f3fec19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:37.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fec102c10 0x7f3fec19b2a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00e9f0 tx=0x7f3fdc00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 shutdown_connections 2024-09-11T00:23:37.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3fd00b5390 0x7f3fd00b7850 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3fec108fa0 0x7f3fec1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3fec108620 0x7f3fec19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 --2- 172.21.15.103:0/3045069358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3fec102c10 0x7f3fec19b2a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:37.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 >> 172.21.15.103:0/3045069358 conn(0x7f3fec0fe660 msgr2=0x7f3fec105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:37.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.319+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 shutdown_connections 2024-09-11T00:23:37.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:37.320+0000 7f3ff0c00640 1 -- 172.21.15.103:0/3045069358 wait complete. 2024-09-11T00:23:37.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:37 smithi117 ceph-mon[35525]: pgmap v414: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T00:23:37.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:37 smithi183 ceph-mon[35934]: pgmap v414: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T00:23:38.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:23:38.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:38 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3045069358' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:38.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:38 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3045069358' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:38.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:38 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3045069358' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:38.539 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:39.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 -- 172.21.15.103:0/2238624222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 msgr2=0x7f21d0111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:39.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 --2- 172.21.15.103:0/2238624222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 0x7f21d0111620 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f21c8009ca0 tx=0x7f21c802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 -- 172.21.15.103:0/2238624222 shutdown_connections 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 --2- 172.21.15.103:0/2238624222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 0x7f21d0111620 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 --2- 172.21.15.103:0/2238624222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 0x7f21d010ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 --2- 172.21.15.103:0/2238624222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d0075540 0x7f21d0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 -- 172.21.15.103:0/2238624222 >> 172.21.15.103:0/2238624222 conn(0x7f21d00fe6c0 msgr2=0x7f21d0100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 -- 172.21.15.103:0/2238624222 shutdown_connections 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.157+0000 7f21d6a00640 1 -- 172.21.15.103:0/2238624222 wait complete. 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 Processor -- start 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 -- start start 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d0075540 0x7f21d019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 0x7f21d019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 0x7f21d01a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:39.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0114e30 con 0x7f21d010f230 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0114cb0 con 0x7f21d0075540 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.158+0000 7f21d6a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0114fb0 con 0x7f21d0075f40 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 0x7f21d01a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 0x7f21d019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 0x7f21d019b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:56954/0 (socket says 172.21.15.103:56954) 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 -- 172.21.15.103:0/534032663 learned_addr learned my addr 172.21.15.103:0/534032663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d5600640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d0075540 0x7f21d019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 -- 172.21.15.103:0/534032663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d0075540 msgr2=0x7f21d019b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d0075540 0x7f21d019b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 -- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 msgr2=0x7f21d01a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 0x7f21d01a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 -- 172.21.15.103:0/534032663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d01a1310 con 0x7f21d0075f40 2024-09-11T00:23:39.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d6000640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 0x7f21d01a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:39.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.159+0000 7f21d4c00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 0x7f21d019b7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00d950 tx=0x7f21bc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:39.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.160+0000 7f21ce000640 1 -- 172.21.15.103:0/534032663 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc017070 con 0x7f21d0075f40 2024-09-11T00:23:39.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.160+0000 7f21ce000640 1 -- 172.21.15.103:0/534032663 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21bc009c30 con 0x7f21d0075f40 2024-09-11T00:23:39.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.160+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d01a1600 con 0x7f21d0075f40 2024-09-11T00:23:39.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.160+0000 7f21ce000640 1 -- 172.21.15.103:0/534032663 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc00f620 con 0x7f21d0075f40 2024-09-11T00:23:39.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.160+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d01a7e90 con 0x7f21d0075f40 2024-09-11T00:23:39.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.161+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d0075920 con 0x7f21d0075f40 2024-09-11T00:23:39.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.163+0000 7f21ce000640 1 -- 172.21.15.103:0/534032663 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f21bc010460 con 0x7f21d0075f40 2024-09-11T00:23:39.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.165+0000 7f21ce000640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f21b80b5410 0x7f21b80b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:39.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.165+0000 7f21ce000640 1 -- 172.21.15.103:0/534032663 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f21bc0e5aa0 con 0x7f21d0075f40 2024-09-11T00:23:39.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.165+0000 7f21d5600640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f21b80b5410 0x7f21b80b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:39.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.166+0000 7f21d5600640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f21b80b5410 0x7f21b80b78d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f21c40095f0 tx=0x7f21c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:39.169 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.168+0000 7f21ce000640 1 -- 172.21.15.103:0/534032663 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f21bc009de0 con 0x7f21d0075f40 2024-09-11T00:23:39.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.393+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21d01a8060 con 0x7f21d0075f40 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: pgmap v415: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:23:39.395 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.394+0000 7f21ce000640 1 -- 172.21.15.103:0/534032663 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11933 (secure 0 0 0) 0x7f21bc0abf20 con 0x7f21d0075f40 2024-09-11T00:23:39.396 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:39.396 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-11T00:23:29:477265+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:23:28.708651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:23:39.396 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f21b80b5410 msgr2=0x7f21b80b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f21b80b5410 0x7f21b80b78d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f21c40095f0 tx=0x7f21c4005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 msgr2=0x7f21d019b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 0x7f21d019b7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00d950 tx=0x7f21bc00de10 comp rx=0 tx=0).stop 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 shutdown_connections 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f21b80b5410 0x7f21b80b78d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21d010f230 0x7f21d01a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21d0075f40 0x7f21d019b7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 --2- 172.21.15.103:0/534032663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21d0075540 0x7f21d019b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 >> 172.21.15.103:0/534032663 conn(0x7f21d00fe6c0 msgr2=0x7f21d00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:39.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 shutdown_connections 2024-09-11T00:23:39.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:39.397+0000 7f21d6a00640 1 -- 172.21.15.103:0/534032663 wait complete. 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: pgmap v415: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:23:39.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: pgmap v415: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:23:39.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:23:40.348 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:23:40.397 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:40 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/534032663' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:40.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:40 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/534032663' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:40.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:40 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/534032663' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:40.719 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:41.299 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 -- 172.21.15.103:0/582863800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0108fa0 msgr2=0x7fd0a010f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 --2- 172.21.15.103:0/582863800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0108fa0 0x7fd0a010f7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd098009ca0 tx=0x7fd09802a8c0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 -- 172.21.15.103:0/582863800 shutdown_connections 2024-09-11T00:23:41.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 --2- 172.21.15.103:0/582863800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0108fa0 0x7fd0a010f7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 --2- 172.21.15.103:0/582863800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0a0108620 0x7fd0a0108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.300 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 --2- 172.21.15.103:0/582863800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a0102c10 0x7fd0a0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 -- 172.21.15.103:0/582863800 >> 172.21.15.103:0/582863800 conn(0x7fd0a00fe660 msgr2=0x7fd0a0100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:41.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 -- 172.21.15.103:0/582863800 shutdown_connections 2024-09-11T00:23:41.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.298+0000 7fd0a6000640 1 -- 172.21.15.103:0/582863800 wait complete. 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.299+0000 7fd0a6000640 1 Processor -- start 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.299+0000 7fd0a6000640 1 -- start start 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0102c10 0x7fd0a00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a0108620 0x7fd0a00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0a0108fa0 0x7fd0a0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a6000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0112cb0 con 0x7fd0a0102c10 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a6000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0112b30 con 0x7fd0a0108620 2024-09-11T00:23:41.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a6000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0112e30 con 0x7fd0a0108fa0 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0102c10 0x7fd0a00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0102c10 0x7fd0a00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33506/0 (socket says 172.21.15.103:33506) 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a4c00640 1 -- 172.21.15.103:0/3472040390 learned_addr learned my addr 172.21.15.103:0/3472040390 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd0a5600640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0a0108fa0 0x7fd0a0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.300+0000 7fd097e00640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a0108620 0x7fd0a00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd0a4c00640 1 -- 172.21.15.103:0/3472040390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0a0108fa0 msgr2=0x7fd0a0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd0a4c00640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0a0108fa0 0x7fd0a0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd0a4c00640 1 -- 172.21.15.103:0/3472040390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a0108620 msgr2=0x7fd0a00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd0a4c00640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a0108620 0x7fd0a00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd0a4c00640 1 -- 172.21.15.103:0/3472040390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a01a7420 con 0x7fd0a0102c10 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd0a4c00640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0102c10 0x7fd0a00754a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e9f0 tx=0x7fd09000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd09e000640 1 -- 172.21.15.103:0/3472040390 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09000cd80 con 0x7fd0a0102c10 2024-09-11T00:23:41.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd09e000640 1 -- 172.21.15.103:0/3472040390 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0900108e0 con 0x7fd0a0102c10 2024-09-11T00:23:41.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd09e000640 1 -- 172.21.15.103:0/3472040390 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090018980 con 0x7fd0a0102c10 2024-09-11T00:23:41.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.301+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a01a7710 con 0x7fd0a0102c10 2024-09-11T00:23:41.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.302+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01a7c00 con 0x7fd0a0102c10 2024-09-11T00:23:41.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.303+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a0102ff0 con 0x7fd0a0102c10 2024-09-11T00:23:41.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.305+0000 7fd09e000640 1 -- 172.21.15.103:0/3472040390 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd090010430 con 0x7fd0a0102c10 2024-09-11T00:23:41.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.307+0000 7fd09e000640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd0840b5390 0x7fd0840b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:41.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.307+0000 7fd09e000640 1 -- 172.21.15.103:0/3472040390 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd0900e62b0 con 0x7fd0a0102c10 2024-09-11T00:23:41.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.307+0000 7fd097e00640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd0840b5390 0x7fd0840b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:41.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.308+0000 7fd097e00640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd0840b5390 0x7fd0840b7850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0076f90 tx=0x7fd08800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:41.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.311+0000 7fd09e000640 1 -- 172.21.15.103:0/3472040390 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd0900ac730 con 0x7fd0a0102c10 2024-09-11T00:23:41.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:41 smithi103 ceph-mon[30578]: pgmap v416: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:23:41.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:41 smithi117 ceph-mon[35525]: pgmap v416: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:23:41.507 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.505+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0a01a8000 con 0x7fd0a0102c10 2024-09-11T00:23:41.508 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.507+0000 7fd09e000640 1 -- 172.21.15.103:0/3472040390 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24865 (secure 0 0 0) 0x7fd0900b1160 con 0x7fd0a0102c10 2024-09-11T00:23:41.509 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:41.509 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":89,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:22:28.671420+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd0840b5390 msgr2=0x7fd0840b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd0840b5390 0x7fd0840b7850 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0076f90 tx=0x7fd08800b390 comp rx=0 tx=0).stop 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0102c10 msgr2=0x7fd0a00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0102c10 0x7fd0a00754a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd09000e9f0 tx=0x7fd09000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 shutdown_connections 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd0840b5390 0x7fd0840b7850 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0a0108fa0 0x7fd0a0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0a0108620 0x7fd0a00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 --2- 172.21.15.103:0/3472040390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0a0102c10 0x7fd0a00754a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 >> 172.21.15.103:0/3472040390 conn(0x7fd0a00fe660 msgr2=0x7fd0a0101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:41.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 shutdown_connections 2024-09-11T00:23:41.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:41.510+0000 7fd0a6000640 1 -- 172.21.15.103:0/3472040390 wait complete. 2024-09-11T00:23:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:41 smithi183 ceph-mon[35934]: pgmap v416: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:23:42.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:42 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3472040390' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:23:42.332 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T00:23:42.431 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:42 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3472040390' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:23:42.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:42 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3472040390' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:23:42.705 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 -- 172.21.15.103:0/2781461853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b4102790 msgr2=0x7fd9b4102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 --2- 172.21.15.103:0/2781461853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b4102790 0x7fd9b4102b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009890 tx=0x7fd9a402a370 comp rx=0 tx=0).stop 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 -- 172.21.15.103:0/2781461853 shutdown_connections 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 --2- 172.21.15.103:0/2781461853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b41042a0 0x7fd9b410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 --2- 172.21.15.103:0/2781461853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b41038e0 0x7fd9b4103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 --2- 172.21.15.103:0/2781461853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b4102790 0x7fd9b4102b90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 -- 172.21.15.103:0/2781461853 >> 172.21.15.103:0/2781461853 conn(0x7fd9b40fe030 msgr2=0x7fd9b4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:42.732 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 -- 172.21.15.103:0/2781461853 shutdown_connections 2024-09-11T00:23:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.731+0000 7fd9b9800640 1 -- 172.21.15.103:0/2781461853 wait complete. 2024-09-11T00:23:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.732+0000 7fd9b9800640 1 Processor -- start 2024-09-11T00:23:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.732+0000 7fd9b9800640 1 -- start start 2024-09-11T00:23:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b9800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4102790 0x7fd9b4076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b41038e0 0x7fd9b4075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b41042a0 0x7fd9b4075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b9800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410e3e0 con 0x7fd9b4102790 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b9800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410e260 con 0x7fd9b41042a0 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b9800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b410e560 con 0x7fd9b41038e0 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4102790 0x7fd9b4076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b41042a0 0x7fd9b4075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4102790 0x7fd9b4076c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55040/0 (socket says 172.21.15.103:55040) 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3311409666 learned_addr learned my addr 172.21.15.103:0/3311409666 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3400640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b41038e0 0x7fd9b4075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3311409666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b41038e0 msgr2=0x7fd9b4075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b41038e0 0x7fd9b4075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3311409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b41042a0 msgr2=0x7fd9b4075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b41042a0 0x7fd9b4075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3311409666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b4076280 con 0x7fd9b4102790 2024-09-11T00:23:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.733+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b41042a0 0x7fd9b4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.734+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4102790 0x7fd9b4076c70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402a960 tx=0x7fd9a4030de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.734+0000 7fd9b0c00640 1 -- 172.21.15.103:0/3311409666 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4037050 con 0x7fd9b4102790 2024-09-11T00:23:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.734+0000 7fd9b0c00640 1 -- 172.21.15.103:0/3311409666 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a4043070 con 0x7fd9b4102790 2024-09-11T00:23:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.734+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b41a2c50 con 0x7fd9b4102790 2024-09-11T00:23:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.734+0000 7fd9b0c00640 1 -- 172.21.15.103:0/3311409666 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4003d10 con 0x7fd9b4102790 2024-09-11T00:23:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.734+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41a3110 con 0x7fd9b4102790 2024-09-11T00:23:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.737+0000 7fd9bde4d740 1 Processor -- start 2024-09-11T00:23:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.737+0000 7fd9bde4d740 1 -- start start 2024-09-11T00:23:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.737+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615f29396b0 con 0x5615f29c6970 2024-09-11T00:23:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615f2928ae0 con 0x5615f29c92a0 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615f292c8f0 con 0x5615f29cbce0 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29c8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55042/0 (socket says 172.21.15.103:55042) 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 -- 172.21.15.103:0/954888415 learned_addr learned my addr 172.21.15.103:0/954888415 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b3400640 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 -- 172.21.15.103:0/954888415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 msgr2=0x5615f29ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 -- 172.21.15.103:0/954888415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 msgr2=0x5615f29cb690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29cb690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.738+0000 7fd9b8e00640 1 -- 172.21.15.103:0/954888415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f28f5780 con 0x5615f29c6970 2024-09-11T00:23:42.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.739+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29c8d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00e8f0 tx=0x7fd9ac0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d4a88e79bd0c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.739+0000 7fd9a9600640 1 -- 172.21.15.103:0/954888415 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac017420 con 0x5615f29c6970 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.739+0000 7fd9a9600640 1 -- 172.21.15.103:0/954888415 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac01c070 con 0x5615f29c6970 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.739+0000 7fd9a9600640 1 -- 172.21.15.103:0/954888415 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac016620 con 0x5615f29c6970 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.739+0000 7fd9b0c00640 1 -- 172.21.15.103:0/3311409666 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd9a4004040 con 0x7fd9b4102790 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 -- 172.21.15.103:0/954888415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 msgr2=0x5615f29c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29c8d60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00e8f0 tx=0x7fd9ac0095c0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 -- 172.21.15.103:0/954888415 shutdown_connections 2024-09-11T00:23:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/954888415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29c8d60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 -- 172.21.15.103:0/954888415 >> 172.21.15.103:0/954888415 conn(0x5615f29c1ac0 msgr2=0x5615f29c2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 -- 172.21.15.103:0/954888415 shutdown_connections 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 -- 172.21.15.103:0/954888415 wait complete. 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.740+0000 7fd9bde4d740 1 -- start start 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 1 Processor -- start 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c92a0 0x5615f29d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615f2860ab0 con 0x5615f29c6970 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5615f28cf570 con 0x5615f29cbce0 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615f29cecc0 con 0x5615f29c92a0 2024-09-11T00:23:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9b0c00640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9980b51f0 0x7fd9980b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.741+0000 7fd9bde4d740 10 client.0 _ll_register_callbacks cb 0x5615f28de100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c92a0 0x5615f29d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b0c00640 1 -- 172.21.15.103:0/3311409666 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd9a4106140 con 0x7fd9b4102790 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c92a0 0x5615f29d07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39178/0 (socket says 172.21.15.103:39178) 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3695334635 learned_addr learned my addr 172.21.15.103:0/3695334635 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9980b51f0 0x7fd9980b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3400640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 msgr2=0x5615f29d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3695334635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 msgr2=0x5615f29d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3e00640 1 -- 172.21.15.103:0/3695334635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f268a790 con 0x5615f29c92a0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b3400640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.742+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9980b51f0 0x7fd9980b76b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00e700 tx=0x7fd9ac009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c92a0 0x5615f29d07e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40cc500 tx=0x7fd9a4003840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb00ee69884e1d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd99e000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd99e000640 1 -- 172.21.15.103:0/3695334635 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40cc340 con 0x5615f29c92a0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac01c070 con 0x5615f29c92a0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd99e000640 1 -- 172.21.15.103:0/3695334635 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd9a4004360 con 0x5615f29c92a0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd99e000640 1 -- 172.21.15.103:0/3695334635 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9a40cbce0 con 0x5615f29c92a0 2024-09-11T00:23:42.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd9bde4d740 10 client.25342 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f2703620 con 0x5615f29c92a0 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.743+0000 7fd99e000640 1 -- 172.21.15.103:0/3695334635 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac01c070 con 0x5615f29c92a0 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd99e000640 1 -- 172.21.15.103:0/3695334635 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd9a40d5670 con 0x5615f29c92a0 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd9bde4d740 10 client.25342 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd9bde4d740 10 client.25342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd9bde4d740 20 client.25342 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f29dba40 0x5615f29dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd9bde4d740 4 client.25342 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615f293b600 con 0x5615f29dba40 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.744+0000 7fd9b3400640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f29dba40 0x5615f29dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.745+0000 7fd9b3400640 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f29dba40 0x5615f29dde60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7bc0bf7b6c6ec server_cookie=3c78b600c007e09e in_seq=0 out_seq=0 2024-09-11T00:23:42.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.745+0000 7fd99e000640 10 client.25342 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd99e000640 1 -- 172.21.15.103:0/3695334635 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615f293b600 con 0x5615f29dba40 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd99e000640 10 client.25342 handle_command_reply: tid=0 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 client.25342 shutdown 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c92a0 msgr2=0x5615f29d07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c92a0 0x5615f29d07e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40cc500 tx=0x7fd9a4003840 comp rx=0 tx=0).stop 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 shutdown_connections 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f29dba40 0x5615f29dde60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c92a0 0x5615f29d07e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3695334635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd99e000640 0 client.25342 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 >> 172.21.15.103:0/3695334635 conn(0x5615f29c1ac0 msgr2=0x5615f29c0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 shutdown_connections 2024-09-11T00:23:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3695334635 wait complete. 2024-09-11T00:23:42.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.746+0000 7fd9bde4d740 20 client.25342 trim_cache size 0 max 16384 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.761+0000 7fd9bde4d740 1 Processor -- start 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9bde4d740 1 -- start start 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41aff00 con 0x5615f29c6970 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ad7d0 con 0x5615f29c92a0 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ab620 con 0x5615f29cbce0 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55066/0 (socket says 172.21.15.103:55066) 2024-09-11T00:23:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3e00640 1 -- 172.21.15.103:0/2108837380 learned_addr learned my addr 172.21.15.103:0/2108837380 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3e00640 1 -- 172.21.15.103:0/2108837380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 msgr2=0x5615f29c2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29c2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3e00640 1 -- 172.21.15.103:0/2108837380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 msgr2=0x5615f29d8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.762+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29d8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.763+0000 7fd9b3e00640 1 -- 172.21.15.103:0/2108837380 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615f29c90d0 con 0x5615f29c6970 2024-09-11T00:23:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.763+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.763+0000 7fd9b3400640 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.763+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29cc7d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4005940 tx=0x7fd9a4030940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb9c6aefc60c6d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.763+0000 7fd9a8c00640 1 -- 172.21.15.103:0/2108837380 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40e1020 con 0x5615f29c6970 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.763+0000 7fd9a8c00640 1 -- 172.21.15.103:0/2108837380 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a4038040 con 0x5615f29c6970 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.763+0000 7fd9a8c00640 1 -- 172.21.15.103:0/2108837380 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40cbec0 con 0x5615f29c6970 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 -- 172.21.15.103:0/2108837380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 msgr2=0x5615f29cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29cc7d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4005940 tx=0x7fd9a4030940 comp rx=0 tx=0).stop 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 -- 172.21.15.103:0/2108837380 shutdown_connections 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29cbce0 0x5615f29c2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29c92a0 0x5615f29d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/2108837380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c6970 0x5615f29cc7d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 -- 172.21.15.103:0/2108837380 >> 172.21.15.103:0/2108837380 conn(0x5615f29c1ac0 msgr2=0x5615f29cc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 -- 172.21.15.103:0/2108837380 shutdown_connections 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 -- 172.21.15.103:0/2108837380 wait complete. 2024-09-11T00:23:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 -- start start 2024-09-11T00:23:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.764+0000 7fd9bde4d740 1 Processor -- start 2024-09-11T00:23:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c6970 0x5615f29d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c92a0 0x5615f29d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9bde4d740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41abc90 con 0x5615f29c92a0 2024-09-11T00:23:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ad100 con 0x5615f29cbce0 2024-09-11T00:23:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9bde4d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ac360 con 0x5615f29c6970 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9bde4d740 10 client.0 _ll_register_callbacks cb 0x7fd9a40edf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c92a0 0x5615f29d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:60094/0 (socket says 172.21.15.103:60094) 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b8e00640 1 -- 172.21.15.103:0/3088506230 learned_addr learned my addr 172.21.15.103:0/3088506230 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c92a0 0x5615f29d6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55072/0 (socket says 172.21.15.103:55072) 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c6970 0x5615f29d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b8e00640 1 -- 172.21.15.103:0/3088506230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c6970 msgr2=0x5615f29d82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.765+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c6970 0x5615f29d82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9b8e00640 1 -- 172.21.15.103:0/3088506230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c92a0 msgr2=0x5615f29d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c92a0 0x5615f29d6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9b8e00640 1 -- 172.21.15.103:0/3088506230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615f29d5b10 con 0x5615f29cbce0 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9b3400640 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c92a0 0x5615f29d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9b8e00640 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d7650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00d7b0 tx=0x7fd9ac00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125049f3610d4f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd99e000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd99e000640 1 -- 172.21.15.103:0/3088506230 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac017d70 con 0x5615f29cbce0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a4038040 con 0x5615f29cbce0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd99e000640 1 -- 172.21.15.103:0/3088506230 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd9ac005050 con 0x5615f29cbce0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd99e000640 1 -- 172.21.15.103:0/3088506230 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac040050 con 0x5615f29cbce0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9bde4d740 10 client.25322 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.766+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5615f2a48bc0 con 0x5615f29cbce0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd99e000640 1 -- 172.21.15.103:0/3088506230 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a4038040 con 0x5615f29cbce0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd99e000640 1 -- 172.21.15.103:0/3088506230 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd9ac00b040 con 0x5615f29cbce0 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9bde4d740 10 client.25322 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9bde4d740 10 client.25322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9bde4d740 20 client.25322 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f2a49400 0x5615f2a4b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:42.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9bde4d740 4 client.25322 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:23:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615f29370b0 con 0x5615f2a49400 2024-09-11T00:23:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f2a49400 0x5615f2a4b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd9b3e00640 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f2a49400 0x5615f2a4b820 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa76dc2319044729 server_cookie=65baee6d900ca4bf in_seq=0 out_seq=0 2024-09-11T00:23:42.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.767+0000 7fd99e000640 10 client.25322 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.779+0000 7fd99e000640 1 -- 172.21.15.103:0/3088506230 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615f29370b0 con 0x5615f2a49400 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.779+0000 7fd99e000640 10 client.25322 handle_command_reply: tid=0 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.779+0000 7fd9bde4d740 1 client.25322 shutdown 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 msgr2=0x5615f29d7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d7650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00d7b0 tx=0x7fd9ac00ef90 comp rx=0 tx=0).stop 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 shutdown_connections 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5615f29cbce0 0x5615f29d7650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615f29c92a0 0x5615f29d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5615f2a49400 0x5615f2a4b820 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 --2- 172.21.15.103:0/3088506230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615f29c6970 0x5615f29d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd99e000640 0 client.25322 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 >> 172.21.15.103:0/3088506230 conn(0x5615f29c1ac0 msgr2=0x5615f29c03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 shutdown_connections 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 1 -- 172.21.15.103:0/3088506230 wait complete. 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.780+0000 7fd9bde4d740 20 client.25322 trim_cache size 0 max 16384 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:23:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:23:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9980b51f0 msgr2=0x7fd9980b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9980b51f0 0x7fd9980b76b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00e700 tx=0x7fd9ac009780 comp rx=0 tx=0).stop 2024-09-11T00:23:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4102790 msgr2=0x7fd9b4076c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4102790 0x7fd9b4076c70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd9a402a960 tx=0x7fd9a4030de0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 shutdown_connections 2024-09-11T00:23:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9980b51f0 0x7fd9980b76b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9b41042a0 0x7fd9b4075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd9b41038e0 0x7fd9b4075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 --2- 172.21.15.103:0/3311409666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9b4102790 0x7fd9b4076c70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 >> 172.21.15.103:0/3311409666 conn(0x7fd9b40fe030 msgr2=0x7fd9b4106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 shutdown_connections 2024-09-11T00:23:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:42.781+0000 7fd9b9800640 1 -- 172.21.15.103:0/3311409666 wait complete. 2024-09-11T00:23:43.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:43 smithi103 ceph-mon[30578]: pgmap v417: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-11T00:23:43.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:43 smithi117 ceph-mon[35525]: pgmap v417: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-11T00:23:43.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:43 smithi183 ceph-mon[35934]: pgmap v417: 129 pgs: 129 active+clean; 648 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-11T00:23:44.301 INFO:teuthology.orchestra.run.smithi103.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T00:23:44.302 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T00:23:44.302 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 6190 2024-09-11T00:23:44.302 DEBUG:tasks.quiescer:Yielding 2024-09-11T00:23:44.303 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T00:23:44.340 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:23:44.341 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T00:23:44.341 INFO:tasks.workunit:timeout=3h 2024-09-11T00:23:44.341 INFO:tasks.workunit:cleanup=True 2024-09-11T00:23:44.341 DEBUG:teuthology.orchestra.run.smithi103:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:44.342 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:23:44.379 INFO:teuthology.orchestra.run.smithi103.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:44.379 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T00:23:44.379 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-11T00:23:44.379 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T00:23:44.380 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T00:23:44.380 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-09-11 00:21:36.922765229 +0000 2024-09-11T00:23:44.380 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-09-11 00:21:36.922765229 +0000 2024-09-11T00:23:44.380 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-09-11 00:23:27.837311112 +0000 2024-09-11T00:23:44.380 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-09-11 00:21:36.922765229 +0000 2024-09-11T00:23:44.380 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T00:23:44.380 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T00:23:44.488 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:23:44.526 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T00:23:44.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:44 smithi103 ceph-mon[30578]: pgmap v418: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-11T00:23:44.738 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:44 smithi183 ceph-mon[35934]: pgmap v418: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-11T00:23:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:44 smithi117 ceph-mon[35525]: pgmap v418: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-09-11T00:23:45.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 -- 172.21.15.103:0/278410541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92801045e0 msgr2=0x7f928010cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:45.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 --2- 172.21.15.103:0/278410541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92801045e0 0x7f928010cad0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f926c009900 tx=0x7f926c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 -- 172.21.15.103:0/278410541 shutdown_connections 2024-09-11T00:23:45.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 --2- 172.21.15.103:0/278410541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f928010d120 0x7f928010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 --2- 172.21.15.103:0/278410541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92801045e0 0x7f928010cad0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 --2- 172.21.15.103:0/278410541 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9280103cc0 0x7f92801040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 -- 172.21.15.103:0/278410541 >> 172.21.15.103:0/278410541 conn(0x7f92800fd700 msgr2=0x7f92800ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 -- 172.21.15.103:0/278410541 shutdown_connections 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.350+0000 7f9286e00640 1 -- 172.21.15.103:0/278410541 wait complete. 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.351+0000 7f9286e00640 1 Processor -- start 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.351+0000 7f9286e00640 1 -- start start 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.351+0000 7f9286e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9280103cc0 0x7f9280199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9286e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92801045e0 0x7f9280199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9286e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f928010d120 0x7f928019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9286e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9280112d90 con 0x7f928010d120 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9286e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9280112c10 con 0x7f92801045e0 2024-09-11T00:23:45.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9286e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9280112f10 con 0x7f9280103cc0 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f928010d120 0x7f928019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f928010d120 0x7f928019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55094/0 (socket says 172.21.15.103:55094) 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 -- 172.21.15.103:0/4201173801 learned_addr learned my addr 172.21.15.103:0/4201173801 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f927fe00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9280103cc0 0x7f9280199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f927f400640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92801045e0 0x7f9280199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 -- 172.21.15.103:0/4201173801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9280103cc0 msgr2=0x7f9280199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9280103cc0 0x7f9280199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 -- 172.21.15.103:0/4201173801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92801045e0 msgr2=0x7f9280199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92801045e0 0x7f9280199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f9284e00640 1 -- 172.21.15.103:0/4201173801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928019f3b0 con 0x7f928010d120 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.352+0000 7f927fe00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9280103cc0 0x7f9280199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.353+0000 7f9284e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f928010d120 0x7f928019ebe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f927800efd0 tx=0x7f927800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.353+0000 7f927cc00640 1 -- 172.21.15.103:0/4201173801 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278003b70 con 0x7f928010d120 2024-09-11T00:23:45.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.353+0000 7f927cc00640 1 -- 172.21.15.103:0/4201173801 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9278003d10 con 0x7f928010d120 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.353+0000 7f927cc00640 1 -- 172.21.15.103:0/4201173801 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92780108a0 con 0x7f928010d120 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.353+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928019f6a0 con 0x7f928010d120 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.353+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92801a5d60 con 0x7f928010d120 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.354+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92801040a0 con 0x7f928010d120 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.356+0000 7f927cc00640 1 -- 172.21.15.103:0/4201173801 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f927801f020 con 0x7f928010d120 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.358+0000 7f927cc00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f92640b5360 0x7f92640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.358+0000 7f927cc00640 1 -- 172.21.15.103:0/4201173801 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f92780e5fe0 con 0x7f928010d120 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.358+0000 7f927fe00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f92640b5360 0x7f92640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.359+0000 7f927fe00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f92640b5360 0x7f92640b7820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9270009fc0 tx=0x7f9270005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:45.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.362+0000 7f927cc00640 1 -- 172.21.15.103:0/4201173801 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f92780acaf0 con 0x7f928010d120 2024-09-11T00:23:45.581 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.579+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92801081f0 con 0x7f928010d120 2024-09-11T00:23:45.582 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.580+0000 7f927cc00640 1 -- 172.21.15.103:0/4201173801 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11933 (secure 0 0 0) 0x7f92780ac3f0 con 0x7f928010d120 2024-09-11T00:23:45.582 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:23:45.582 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-11T00:23:29:477265+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:23:28.708651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:23:45.582 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-11T00:23:45.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f92640b5360 msgr2=0x7f92640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:45.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f92640b5360 0x7f92640b7820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9270009fc0 tx=0x7f9270005f90 comp rx=0 tx=0).stop 2024-09-11T00:23:45.584 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f928010d120 msgr2=0x7f928019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f928010d120 0x7f928019ebe0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f927800efd0 tx=0x7f927800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 shutdown_connections 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f92640b5360 0x7f92640b7820 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f928010d120 0x7f928019ebe0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f92801045e0 0x7f9280199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 --2- 172.21.15.103:0/4201173801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9280103cc0 0x7f9280199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.582+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 >> 172.21.15.103:0/4201173801 conn(0x7f92800fd700 msgr2=0x7f92800fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.583+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 shutdown_connections 2024-09-11T00:23:45.585 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:45.583+0000 7f9286e00640 1 -- 172.21.15.103:0/4201173801 wait complete. 2024-09-11T00:23:45.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:45 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4201173801' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:45.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:45 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4201173801' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:46.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:45 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4201173801' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:23:46.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T00:20:28.229329+0000', 'modified': '2024-09-11T00:23:28.708651+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24656, 'mds_1': 24643, 'mds_2': 24628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24599': {'gid': 24599, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6835/2973217634', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 2973217634}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 2973217634}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24620': {'gid': 24620, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.117:6837/2700270511', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 2700270511}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2700270511}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24622': {'gid': 24622, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/3946403859', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 3946403859}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3946403859}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24628': {'gid': 24628, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.183:6837/335217865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 335217865}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 335217865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24643': {'gid': 24643, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.183:6839/424027451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 424027451}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 424027451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24656': {'gid': 24656, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.117:6839/3769588373', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6838', 'nonce': 3769588373}, {'type': 'v1', 'addr': '172.21.15.117:6839', 'nonce': 3769588373}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24656, 'qdb_cluster': [24628, 24643, 24656]} 2024-09-11T00:23:46.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-11T00:23:46.402 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T00:23:46.814 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:46.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:46 smithi103 ceph-mon[30578]: pgmap v419: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T00:23:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:46 smithi117 ceph-mon[35525]: pgmap v419: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T00:23:47.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:46 smithi183 ceph-mon[35934]: pgmap v419: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-09-11T00:23:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 -- 172.21.15.103:0/89626515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46641087d0 msgr2=0x7f4664108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 --2- 172.21.15.103:0/89626515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46641087d0 0x7f4664108c10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f464c009900 tx=0x7f464c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 -- 172.21.15.103:0/89626515 shutdown_connections 2024-09-11T00:23:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 --2- 172.21.15.103:0/89626515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4664109150 0x7f466410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 --2- 172.21.15.103:0/89626515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46641087d0 0x7f4664108c10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.552 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 --2- 172.21.15.103:0/89626515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664102dc0 0x7f46641031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 -- 172.21.15.103:0/89626515 >> 172.21.15.103:0/89626515 conn(0x7f46640fe830 msgr2=0x7f4664100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 -- 172.21.15.103:0/89626515 shutdown_connections 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.445+0000 7f466aa00640 1 -- 172.21.15.103:0/89626515 wait complete. 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.446+0000 7f466aa00640 1 Processor -- start 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.446+0000 7f466aa00640 1 -- start start 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f466aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4664102dc0 0x7f466419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f466aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641087d0 0x7f466419ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f466aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664109150 0x7f46641a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f466aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4664112e60 con 0x7f4664102dc0 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f466aa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4664112ce0 con 0x7f46641087d0 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f466aa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4664112fe0 con 0x7f4664109150 2024-09-11T00:23:47.553 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f4663e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4664102dc0 0x7f466419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f4663e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4664102dc0 0x7f466419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55104/0 (socket says 172.21.15.103:55104) 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f4663e00640 1 -- 172.21.15.103:0/4173301784 learned_addr learned my addr 172.21.15.103:0/4173301784 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f465be00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641087d0 0x7f466419ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f4668a00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664109150 0x7f46641a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.447+0000 7f4663e00640 1 -- 172.21.15.103:0/4173301784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664109150 msgr2=0x7f46641a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4663e00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664109150 0x7f46641a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4663e00640 1 -- 172.21.15.103:0/4173301784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641087d0 msgr2=0x7f466419ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4663e00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641087d0 0x7f466419ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4663e00640 1 -- 172.21.15.103:0/4173301784 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46641a1570 con 0x7f4664102dc0 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4663e00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4664102dc0 0x7f466419b4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f465400d950 tx=0x7f465400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4661600640 1 -- 172.21.15.103:0/4173301784 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4654017030 con 0x7f4664102dc0 2024-09-11T00:23:47.554 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4661600640 1 -- 172.21.15.103:0/4173301784 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4654004020 con 0x7f4664102dc0 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46641a1860 con 0x7f4664102dc0 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f4661600640 1 -- 172.21.15.103:0/4173301784 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46540047a0 con 0x7f4664102dc0 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.448+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46641a80f0 con 0x7f4664102dc0 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.451+0000 7f466d480740 1 Processor -- start 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.451+0000 7f466d480740 1 -- start start 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f4661600640 1 -- 172.21.15.103:0/4173301784 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4654009a30 con 0x7f4664102dc0 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f466d480740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a331620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f466d480740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f466d480740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f466d480740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e3a2a8d50 con 0x559e3a33daf0 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f466d480740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e3a298180 con 0x559e3a331b60 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f466d480740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e3a29bf90 con 0x559e3a331240 2024-09-11T00:23:47.555 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f4668a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a331620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f465be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f4668a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a331620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39260/0 (socket says 172.21.15.103:39260) 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f4668a00640 1 -- 172.21.15.103:0/1795469463 learned_addr learned my addr 172.21.15.103:0/1795469463 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.452+0000 7f4663e00640 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4668a00640 1 -- 172.21.15.103:0/1795469463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 msgr2=0x559e3a33d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4668a00640 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4668a00640 1 -- 172.21.15.103:0/1795469463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 msgr2=0x559e3a33fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4668a00640 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4668a00640 1 -- 172.21.15.103:0/1795469463 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3a262a70 con 0x559e3a331240 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4668a00640 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a331620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f465c009fc0 tx=0x7f465c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8e93af36c990376 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4659200640 1 -- 172.21.15.103:0/1795469463 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c011700 con 0x559e3a331240 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4659200640 1 -- 172.21.15.103:0/1795469463 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c011d50 con 0x559e3a331240 2024-09-11T00:23:47.556 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.453+0000 7f4659200640 1 -- 172.21.15.103:0/1795469463 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c019dd0 con 0x559e3a331240 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f4661600640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46480b5360 0x7f46480b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f4661600640 1 -- 172.21.15.103:0/4173301784 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f46540e5900 con 0x7f4664102dc0 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f4663e00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46480b5360 0x7f46480b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 -- 172.21.15.103:0/1795469463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 msgr2=0x559e3a331620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a331620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f465c009fc0 tx=0x7f465c010860 comp rx=0 tx=0).stop 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 -- 172.21.15.103:0/1795469463 shutdown_connections 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 --2- 172.21.15.103:0/1795469463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a331620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 -- 172.21.15.103:0/1795469463 >> 172.21.15.103:0/1795469463 conn(0x559e3a32f020 msgr2=0x559e3a330210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.454+0000 7f466d480740 1 -- 172.21.15.103:0/1795469463 shutdown_connections 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.455+0000 7f466d480740 1 -- 172.21.15.103:0/1795469463 wait complete. 2024-09-11T00:23:47.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.455+0000 7f4663e00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46480b5360 0x7f46480b7820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46540e33e0 tx=0x7f4654000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.455+0000 7f466d480740 1 -- start start 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.455+0000 7f466d480740 1 Processor -- start 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f466d480740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a33aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f466d480740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f466d480740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f466d480740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e3a0f95f0 con 0x559e3a33daf0 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f466d480740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e3a233460 con 0x559e3a331b60 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f466d480740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e3a340840 con 0x559e3a331240 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f466d480740 10 client.0 _ll_register_callbacks cb 0x559e3a243210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f4668a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a33aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f4668a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f4668a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55110/0 (socket says 172.21.15.103:55110) 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f4668a00640 1 -- 172.21.15.103:0/3323931791 learned_addr learned my addr 172.21.15.103:0/3323931791 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:47.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.456+0000 7f465be00640 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4668a00640 1 -- 172.21.15.103:0/3323931791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 msgr2=0x559e3a33aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4668a00640 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a33aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4668a00640 1 -- 172.21.15.103:0/3323931791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 msgr2=0x559e3a33aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4668a00640 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4668a00640 1 -- 172.21.15.103:0/3323931791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e39ff6540 con 0x559e3a33daf0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4668a00640 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33b530 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f465c019830 tx=0x7f465c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74664fb6005eec40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4651600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4651600640 1 -- 172.21.15.103:0/3323931791 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c01f040 con 0x559e3a33daf0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e3a235a70 con 0x559e3a33daf0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4651600640 1 -- 172.21.15.103:0/3323931791 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f465c003b70 con 0x559e3a33daf0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f4651600640 1 -- 172.21.15.103:0/3323931791 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f465c003d70 con 0x559e3a33daf0 2024-09-11T00:23:47.559 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f466d480740 10 client.15330 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.457+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e3a2506f0 con 0x559e3a33daf0 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f4651600640 1 -- 172.21.15.103:0/3323931791 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e3a235a70 con 0x559e3a33daf0 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f4651600640 1 -- 172.21.15.103:0/3323931791 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f465c011570 con 0x559e3a33daf0 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f466d480740 10 client.15330 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f466d480740 10 client.15330 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f466d480740 20 client.15330 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f466d480740 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a34e410 0x559e3a34f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f466d480740 4 client.15330 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.458+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 --> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e3a2aaca0 con 0x559e3a34e410 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.459+0000 7f465be00640 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a34e410 0x559e3a34f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.459+0000 7f465be00640 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a34e410 0x559e3a34f880 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5765d1675d97deaa server_cookie=b3ae149884dec314 in_seq=0 out_seq=0 2024-09-11T00:23:47.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.459+0000 7f4651600640 10 client.15330 ms_handle_connect on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f4651600640 1 -- 172.21.15.103:0/3323931791 <== mds.2 v2:172.21.15.183:6836/335217865 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559e3a2aaca0 con 0x559e3a34e410 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f4651600640 10 client.15330 handle_command_reply: tid=0 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 client.15330 shutdown 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 msgr2=0x559e3a33b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33b530 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f465c019830 tx=0x7f465c000f40 comp rx=0 tx=0).stop 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 shutdown_connections 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a34e410 0x559e3a34f880 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a33b530 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331b60 0x559e3a33aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 --2- 172.21.15.103:0/3323931791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a33aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f4651600640 0 client.15330 ms_handle_reset on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 >> 172.21.15.103:0/3323931791 conn(0x559e3a32f020 msgr2=0x559e3a334c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:47.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 shutdown_connections 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 1 -- 172.21.15.103:0/3323931791 wait complete. 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.460+0000 7f466d480740 20 client.15330 trim_cache size 0 max 16384 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 Processor -- start 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 -- start start 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331240 0x559e3a34eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331b60 0x559e3a3307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a337cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e3a34dbc0 con 0x559e3a33daf0 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e3a34e290 con 0x559e3a331240 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.479+0000 7f466d480740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e3a34e960 con 0x559e3a331b60 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a337cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a337cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55114/0 (socket says 172.21.15.103:55114) 2024-09-11T00:23:47.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 -- 172.21.15.103:0/3375885888 learned_addr learned my addr 172.21.15.103:0/3375885888 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331240 0x559e3a34eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f465be00640 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331b60 0x559e3a3307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 -- 172.21.15.103:0/3375885888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331b60 msgr2=0x559e3a3307d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331b60 0x559e3a3307d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 -- 172.21.15.103:0/3375885888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331240 msgr2=0x559e3a34eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331240 0x559e3a34eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f4668a00640 1 -- 172.21.15.103:0/3375885888 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e3a3294b0 con 0x559e3a33daf0 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.480+0000 7f465be00640 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331b60 0x559e3a3307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.481+0000 7f4668a00640 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a337cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f465c009850 tx=0x7f465c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3139864caa88113 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.481+0000 7f4653e00640 1 -- 172.21.15.103:0/3375885888 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c011c50 con 0x559e3a33daf0 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.481+0000 7f4653e00640 1 -- 172.21.15.103:0/3375885888 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c011df0 con 0x559e3a33daf0 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.481+0000 7f4653e00640 1 -- 172.21.15.103:0/3375885888 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c011740 con 0x559e3a33daf0 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.481+0000 7f466d480740 1 -- 172.21.15.103:0/3375885888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 msgr2=0x559e3a337cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.481+0000 7f466d480740 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a337cc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f465c009850 tx=0x7f465c013040 comp rx=0 tx=0).stop 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 -- 172.21.15.103:0/3375885888 shutdown_connections 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a33daf0 0x559e3a337cc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331b60 0x559e3a3307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 --2- 172.21.15.103:0/3375885888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a331240 0x559e3a34eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 -- 172.21.15.103:0/3375885888 >> 172.21.15.103:0/3375885888 conn(0x559e3a32f020 msgr2=0x559e3a3300c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 -- 172.21.15.103:0/3375885888 shutdown_connections 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 -- 172.21.15.103:0/3375885888 wait complete. 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 -- start start 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.482+0000 7f466d480740 1 Processor -- start 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f466d480740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a351d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f466d480740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a331b60 0x559e3a3522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f466d480740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a33daf0 0x559e3a3527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f466d480740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559e3a34f700 con 0x559e3a331b60 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f466d480740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559e3a34f030 con 0x559e3a33daf0 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f466d480740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559e3a34fdd0 con 0x559e3a331240 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f466d480740 10 client.0 _ll_register_callbacks cb 0x7f465c019b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f465be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a331b60 0x559e3a3522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f465be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a331b60 0x559e3a3522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55118/0 (socket says 172.21.15.103:55118) 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f465be00640 1 -- 172.21.15.103:0/1780456984 learned_addr learned my addr 172.21.15.103:0/1780456984 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f4668a00640 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a351d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.483+0000 7f4668a00640 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a33daf0 0x559e3a3527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f465be00640 1 -- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 msgr2=0x559e3a351d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f465be00640 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a351d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f465be00640 1 -- 172.21.15.103:0/1780456984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a33daf0 msgr2=0x559e3a3527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f465be00640 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a33daf0 0x559e3a3527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f465be00640 1 -- 172.21.15.103:0/1780456984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e3a357710 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f465be00640 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a331b60 0x559e3a3522b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f464c02a9b0 tx=0x7f464c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae0627a3743a1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f4651600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f4651600640 1 -- 172.21.15.103:0/1780456984 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464c035040 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f465c011df0 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f4651600640 1 -- 172.21.15.103:0/1780456984 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f464c005590 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.484+0000 7f4651600640 1 -- 172.21.15.103:0/1780456984 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f464c004770 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 10 client.15336 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e3a3bb340 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f4651600640 1 -- 172.21.15.103:0/1780456984 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f465c011df0 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f4651600640 1 -- 172.21.15.103:0/1780456984 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f464c003b50 con 0x559e3a331b60 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 10 client.15336 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 10 client.15336 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T00:23:47.566 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 20 client.15336 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a3bff20 0x559e3a3bec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 4 client.15336 mds_command: new command op to 24628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.485+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 --> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] -- command(tid 0: {"prefix": "status"}) -- 0x559e3a2a6750 con 0x559e3a3bff20 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f4668a00640 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a3bff20 0x559e3a3bec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f4668a00640 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a3bff20 0x559e3a3bec30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cd3f3896ae47387 server_cookie=f80c839b9afa6e05 in_seq=0 out_seq=0 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f4651600640 10 client.15336 ms_handle_connect on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f4651600640 1 -- 172.21.15.103:0/1780456984 <== mds.2 v2:172.21.15.183:6836/335217865 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559e3a2a6750 con 0x559e3a3bff20 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f4651600640 10 client.15336 handle_command_reply: tid=0 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 client.15336 shutdown 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a331b60 msgr2=0x559e3a3522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a331b60 0x559e3a3522b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f464c02a9b0 tx=0x7f464c0045c0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 shutdown_connections 2024-09-11T00:23:47.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:6836/335217865,v1:172.21.15.183:6837/335217865] conn(0x559e3a3bff20 0x559e3a3bec30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559e3a33daf0 0x559e3a3527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f4651600640 0 client.15336 ms_handle_reset on v2:172.21.15.183:6836/335217865 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559e3a331b60 0x559e3a3522b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.486+0000 7f466d480740 1 --2- 172.21.15.103:0/1780456984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559e3a331240 0x559e3a351d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 >> 172.21.15.103:0/1780456984 conn(0x559e3a32f020 msgr2=0x559e3a348ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 shutdown_connections 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466d480740 1 -- 172.21.15.103:0/1780456984 wait complete. 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:23:47.568 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 2, 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24628, 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 144.36722432299999, 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 89, 2024-09-11T00:23:47.569 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 230.15521349599999 2024-09-11T00:23:47.570 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:23:47.571 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466d480740 20 client.15336 trim_cache size 0 max 16384 2024-09-11T00:23:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46480b5360 msgr2=0x7f46480b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.572 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466aa00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46480b5360 0x7f46480b7820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f46540e33e0 tx=0x7f4654000bc0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4664102dc0 msgr2=0x7f466419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:47.573 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466aa00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4664102dc0 0x7f466419b4c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f465400d950 tx=0x7f465400de10 comp rx=0 tx=0).stop 2024-09-11T00:23:47.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 shutdown_connections 2024-09-11T00:23:47.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466aa00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46480b5360 0x7f46480b7820 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.487+0000 7f466aa00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4664109150 0x7f46641a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.488+0000 7f466aa00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46641087d0 0x7f466419ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.574 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.488+0000 7f466aa00640 1 --2- 172.21.15.103:0/4173301784 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4664102dc0 0x7f466419b4c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:47.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.488+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 >> 172.21.15.103:0/4173301784 conn(0x7f46640fe830 msgr2=0x7f4664075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:47.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.488+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 shutdown_connections 2024-09-11T00:23:47.575 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:47.488+0000 7f466aa00640 1 -- 172.21.15.103:0/4173301784 wait complete. 2024-09-11T00:23:48.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T00:23:48.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:48 smithi103 ceph-mon[30578]: pgmap v420: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-11T00:23:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:48 smithi183 ceph-mon[35934]: pgmap v420: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-11T00:23:48.781 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:48 smithi117 ceph-mon[35525]: pgmap v420: 129 pgs: 129 active+clean; 653 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-11T00:23:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 -- 172.21.15.103:0/1337259681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb810b1a0 msgr2=0x7f0eb81119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 --2- 172.21.15.103:0/1337259681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb810b1a0 0x7f0eb81119a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009900 tx=0x7f0eb002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 -- 172.21.15.103:0/1337259681 shutdown_connections 2024-09-11T00:23:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 --2- 172.21.15.103:0/1337259681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb810b1a0 0x7f0eb81119a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 --2- 172.21.15.103:0/1337259681 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb810a820 0x7f0eb810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 --2- 172.21.15.103:0/1337259681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb8104e10 0x7f0eb81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 -- 172.21.15.103:0/1337259681 >> 172.21.15.103:0/1337259681 conn(0x7f0eb8100840 msgr2=0x7f0eb8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 -- 172.21.15.103:0/1337259681 shutdown_connections 2024-09-11T00:23:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.347+0000 7f0ebca00640 1 -- 172.21.15.103:0/1337259681 wait complete. 2024-09-11T00:23:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 Processor -- start 2024-09-11T00:23:49.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 -- start start 2024-09-11T00:23:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8104e10 0x7f0eb819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb810a820 0x7f0eb819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb810b1a0 0x7f0eb81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8114eb0 con 0x7f0eb8104e10 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8114d30 con 0x7f0eb810a820 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0ebca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb8115030 con 0x7f0eb810b1a0 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.348+0000 7f0eb6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8104e10 0x7f0eb819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8104e10 0x7f0eb819b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55130/0 (socket says 172.21.15.103:55130) 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 -- 172.21.15.103:0/3311329253 learned_addr learned my addr 172.21.15.103:0/3311329253 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb7200640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb810b1a0 0x7f0eb81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eade00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb810a820 0x7f0eb819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 -- 172.21.15.103:0/3311329253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb810b1a0 msgr2=0x7f0eb81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb810b1a0 0x7f0eb81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 -- 172.21.15.103:0/3311329253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb810a820 msgr2=0x7f0eb819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb810a820 0x7f0eb819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 -- 172.21.15.103:0/3311329253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb81a1330 con 0x7f0eb8104e10 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.349+0000 7f0eb6800640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8104e10 0x7f0eb819b280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800eb90 tx=0x7f0ea800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.350+0000 7f0eafe00640 1 -- 172.21.15.103:0/3311329253 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8004d20 con 0x7f0eb8104e10 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.350+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81a1620 con 0x7f0eb8104e10 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.350+0000 7f0eafe00640 1 -- 172.21.15.103:0/3311329253 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea800ee90 con 0x7f0eb8104e10 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.350+0000 7f0eafe00640 1 -- 172.21.15.103:0/3311329253 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8010670 con 0x7f0eb8104e10 2024-09-11T00:23:49.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.350+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a7ce0 con 0x7f0eb8104e10 2024-09-11T00:23:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 Processor -- start 2024-09-11T00:23:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 -- start start 2024-09-11T00:23:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0eafe00640 1 -- 172.21.15.103:0/3311329253 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0ea8005020 con 0x7f0eb8104e10 2024-09-11T00:23:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e1521c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e15228770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e1522b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e15193d50 con 0x562e15228cb0 2024-09-11T00:23:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e15183180 con 0x562e1521c400 2024-09-11T00:23:49.356 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.353+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e15186f90 con 0x562e1521cd20 2024-09-11T00:23:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e1522b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e15228770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e1522b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55144/0 (socket says 172.21.15.103:55144) 2024-09-11T00:23:49.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eade00640 1 -- 172.21.15.103:0/2687052661 learned_addr learned my addr 172.21.15.103:0/2687052661 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e1521c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb7200640 1 -- 172.21.15.103:0/2687052661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 msgr2=0x562e15228770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2687052661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e15228770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb7200640 1 -- 172.21.15.103:0/2687052661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 msgr2=0x562e1522b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2687052661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e1522b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb7200640 1 -- 172.21.15.103:0/2687052661 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1514da70 con 0x562e1521c400 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.354+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2687052661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e1521c7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002a910 tx=0x7f0eb0004130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4658660591c39cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.355+0000 7f0ea7400640 1 -- 172.21.15.103:0/2687052661 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0040070 con 0x562e1521c400 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.355+0000 7f0eafe00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e9c0b5410 0x7f0e9c0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.355+0000 7f0ea7400640 1 -- 172.21.15.103:0/2687052661 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb00054d0 con 0x562e1521c400 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.355+0000 7f0ea7400640 1 -- 172.21.15.103:0/2687052661 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0035040 con 0x562e1521c400 2024-09-11T00:23:49.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.355+0000 7f0eafe00640 1 -- 172.21.15.103:0/3311329253 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0ea80e6580 con 0x7f0eb8104e10 2024-09-11T00:23:49.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.355+0000 7f0eb6800640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e9c0b5410 0x7f0e9c0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2687052661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 msgr2=0x562e1521c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2687052661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e1521c7e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0eb002a910 tx=0x7f0eb0004130 comp rx=0 tx=0).stop 2024-09-11T00:23:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0eb6800640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e9c0b5410 0x7f0e9c0b78d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80ac8e0 tx=0x7f0ea80ec060 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2687052661 shutdown_connections 2024-09-11T00:23:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2687052661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e1522b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2687052661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e15228770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2687052661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e1521c7e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2687052661 >> 172.21.15.103:0/2687052661 conn(0x562e1521a230 msgr2=0x562e1521b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2687052661 shutdown_connections 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.356+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2687052661 wait complete. 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 -- start start 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 Processor -- start 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e15225c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e152261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e152266e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e14fe45f0 con 0x562e15228cb0 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e1511e460 con 0x562e1521c400 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e1522b9a0 con 0x562e1521cd20 2024-09-11T00:23:49.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.357+0000 7f0ebfe17740 10 client.0 _ll_register_callbacks cb 0x562e1512e210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e15225c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e152266e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e152266e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55156/0 (socket says 172.21.15.103:55156) 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eade00640 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e152261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 -- 172.21.15.103:0/817523711 learned_addr learned my addr 172.21.15.103:0/817523711 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 -- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 msgr2=0x562e152261a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e152261a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 -- 172.21.15.103:0/817523711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 msgr2=0x562e15225c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e15225c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 -- 172.21.15.103:0/817523711 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e14ee1540 con 0x562e15228cb0 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eade00640 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e152261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:49.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.358+0000 7f0eb7200640 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e152266e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0006010 tx=0x7f0eb002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c59ec98363c649 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0e9be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0e9be00640 1 -- 172.21.15.103:0/817523711 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0035070 con 0x562e15228cb0 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0e9be00640 1 -- 172.21.15.103:0/817523711 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0eb00051e0 con 0x562e15228cb0 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e15120a70 con 0x562e15228cb0 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0e9be00640 1 -- 172.21.15.103:0/817523711 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0040040 con 0x562e15228cb0 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0ebfe17740 10 client.15348 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e1513b6f0 con 0x562e15228cb0 2024-09-11T00:23:49.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.359+0000 7f0e9be00640 1 -- 172.21.15.103:0/817523711 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e15120a70 con 0x562e15228cb0 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0e9be00640 1 -- 172.21.15.103:0/817523711 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0eb00314a0 con 0x562e15228cb0 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0ebfe17740 10 client.15348 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0ebfe17740 10 client.15348 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0ebfe17740 20 client.15348 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152386c0 0x562e1523a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0ebfe17740 4 client.15348 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 --> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e15195ca0 con 0x562e152386c0 2024-09-11T00:23:49.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.360+0000 7f0eade00640 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152386c0 0x562e1523a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.361+0000 7f0eade00640 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152386c0 0x562e1523a860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c7b2aba1277c5cde server_cookie=138cb3f626c33cd0 in_seq=0 out_seq=0 2024-09-11T00:23:49.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.361+0000 7f0e9be00640 10 client.15348 ms_handle_connect on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:49.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0e9be00640 1 -- 172.21.15.103:0/817523711 <== mds.1 v2:172.21.15.183:6838/424027451 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562e15195ca0 con 0x562e152386c0 2024-09-11T00:23:49.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0e9be00640 10 client.15348 handle_command_reply: tid=0 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 client.15348 shutdown 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 msgr2=0x562e152266e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e152266e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0006010 tx=0x7f0eb002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 shutdown_connections 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152386c0 0x562e1523a860 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e15228cb0 0x562e152266e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521cd20 0x562e152261a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/817523711 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e1521c400 0x562e15225c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 >> 172.21.15.103:0/817523711 conn(0x562e1521a230 msgr2=0x562e1521fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0e9be00640 0 client.15348 ms_handle_reset on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 shutdown_connections 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.362+0000 7f0ebfe17740 1 -- 172.21.15.103:0/817523711 wait complete. 2024-09-11T00:23:49.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.363+0000 7f0ebfe17740 20 client.15348 trim_cache size 0 max 16384 2024-09-11T00:23:49.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.381+0000 7f0ebfe17740 1 Processor -- start 2024-09-11T00:23:49.386 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.381+0000 7f0ebfe17740 1 -- start start 2024-09-11T00:23:49.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e152393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1521b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e15222e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e15238ba0 con 0x562e1521cd20 2024-09-11T00:23:49.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e15239270 con 0x562e15228cb0 2024-09-11T00:23:49.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e15239940 con 0x562e1521c400 2024-09-11T00:23:49.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1521b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1521b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55170/0 (socket says 172.21.15.103:55170) 2024-09-11T00:23:49.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 -- 172.21.15.103:0/1590374184 learned_addr learned my addr 172.21.15.103:0/1590374184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:49.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eb7200640 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e152393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eb7200640 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e15222e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 -- 172.21.15.103:0/1590374184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 msgr2=0x562e152393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e152393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 -- 172.21.15.103:0/1590374184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 msgr2=0x562e15222e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e15222e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eade00640 1 -- 172.21.15.103:0/1590374184 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e1521acb0 con 0x562e1521cd20 2024-09-11T00:23:49.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.382+0000 7f0eb7200640 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e15222e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:49.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.383+0000 7f0eade00640 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1521b7e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562e1522baf0 tx=0x7f0ea000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825e85420d859c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:49.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.383+0000 7f0ea6a00640 1 -- 172.21.15.103:0/1590374184 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0003cc0 con 0x562e1521cd20 2024-09-11T00:23:49.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.383+0000 7f0ea6a00640 1 -- 172.21.15.103:0/1590374184 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea0003e60 con 0x562e1521cd20 2024-09-11T00:23:49.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.383+0000 7f0ea6a00640 1 -- 172.21.15.103:0/1590374184 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0024470 con 0x562e1521cd20 2024-09-11T00:23:49.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.383+0000 7f0ebfe17740 1 -- 172.21.15.103:0/1590374184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 msgr2=0x562e1521b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.383+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1521b7e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x562e1522baf0 tx=0x7f0ea000b390 comp rx=0 tx=0).stop 2024-09-11T00:23:49.390 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 -- 172.21.15.103:0/1590374184 shutdown_connections 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e15222e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1521b7e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/1590374184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e152393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 -- 172.21.15.103:0/1590374184 >> 172.21.15.103:0/1590374184 conn(0x562e1521a230 msgr2=0x562e1521b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 -- 172.21.15.103:0/1590374184 shutdown_connections 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 -- 172.21.15.103:0/1590374184 wait complete. 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 -- start start 2024-09-11T00:23:49.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.384+0000 7f0ebfe17740 1 Processor -- start 2024-09-11T00:23:49.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e15243550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1523cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0ebfe17740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e1523d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562e1523a6e0 con 0x562e1521cd20 2024-09-11T00:23:49.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562e1523a010 con 0x562e15228cb0 2024-09-11T00:23:49.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0ebfe17740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e1523adb0 con 0x562e1521c400 2024-09-11T00:23:49.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0ebfe17740 10 client.0 _ll_register_callbacks cb 0x7f0eb00045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:49.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0eade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1523cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0eade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1523cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55178/0 (socket says 172.21.15.103:55178) 2024-09-11T00:23:49.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0eade00640 1 -- 172.21.15.103:0/2482340426 learned_addr learned my addr 172.21.15.103:0/2482340426 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:49.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e15243550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.385+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e1523d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0eade00640 1 -- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 msgr2=0x562e15243550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0eade00640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e15243550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0eade00640 1 -- 172.21.15.103:0/2482340426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 msgr2=0x562e1523d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:49.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0eade00640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e1523d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0eade00640 1 -- 172.21.15.103:0/2482340426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e1523efc0 con 0x562e1521cd20 2024-09-11T00:23:49.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e15243550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:49.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0eade00640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1523cd10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005940 tx=0x7f0ea0003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d523f54182e8d48d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:49.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0e9be00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:49.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0e9be00640 1 -- 172.21.15.103:0/2482340426 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0031020 con 0x562e1521cd20 2024-09-11T00:23:49.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0e9be00640 1 -- 172.21.15.103:0/2482340426 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0ea0004930 con 0x562e1521cd20 2024-09-11T00:23:49.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea0003e60 con 0x562e1521cd20 2024-09-11T00:23:49.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0e9be00640 1 -- 172.21.15.103:0/2482340426 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0ea003c050 con 0x562e1521cd20 2024-09-11T00:23:49.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.386+0000 7f0ebfe17740 10 client.15354 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:49.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562e152a6050 con 0x562e1521cd20 2024-09-11T00:23:49.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0e9be00640 1 -- 172.21.15.103:0/2482340426 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea0003e60 con 0x562e1521cd20 2024-09-11T00:23:49.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0e9be00640 1 -- 172.21.15.103:0/2482340426 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0ea0024930 con 0x562e1521cd20 2024-09-11T00:23:49.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0ebfe17740 10 client.15354 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:49.400 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0ebfe17740 10 client.15354 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T00:23:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0ebfe17740 20 client.15354 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152aa920 0x562e152a9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0ebfe17740 4 client.15354 mds_command: new command op to 24643 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:23:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.387+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 --> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] -- command(tid 0: {"prefix": "status"}) -- 0x562e15191750 con 0x562e152aa920 2024-09-11T00:23:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152aa920 0x562e152a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0eb7200640 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152aa920 0x562e152a9630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=526f7de1e1a6647c server_cookie=bf293e755ce4ffbd in_seq=0 out_seq=0 2024-09-11T00:23:49.401 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0e9be00640 10 client.15354 ms_handle_connect on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:49.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0e9be00640 1 -- 172.21.15.103:0/2482340426 <== mds.1 v2:172.21.15.183:6838/424027451 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562e15191750 con 0x562e152aa920 2024-09-11T00:23:49.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0e9be00640 10 client.15354 handle_command_reply: tid=0 2024-09-11T00:23:49.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0ebfe17740 1 client.15354 shutdown 2024-09-11T00:23:49.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 msgr2=0x562e1523cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.388+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1523cd10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005940 tx=0x7f0ea0003cb0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 shutdown_connections 2024-09-11T00:23:49.402 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:6838/424027451,v1:172.21.15.183:6839/424027451] conn(0x562e152aa920 0x562e152a9630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562e15228cb0 0x562e1523d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562e1521cd20 0x562e1523cd10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 --2- 172.21.15.103:0/2482340426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e1521c400 0x562e15243550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 >> 172.21.15.103:0/2482340426 conn(0x562e1521a230 msgr2=0x562e15227290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0e9be00640 0 client.15354 ms_handle_reset on v2:172.21.15.183:6838/424027451 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 shutdown_connections 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 1 -- 172.21.15.103:0/2482340426 wait complete. 2024-09-11T00:23:49.403 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebfe17740 20 client.15354 trim_cache size 0 max 16384 2024-09-11T00:23:49.404 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:23:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:23:49.404 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 1, 2024-09-11T00:23:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24643, 2024-09-11T00:23:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:23:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:23:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:23:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 147.28340007899999, 2024-09-11T00:23:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-11T00:23:49.405 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:23:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 89, 2024-09-11T00:23:49.406 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 226.71859158999999 2024-09-11T00:23:49.406 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:23:49.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e9c0b5410 msgr2=0x7f0e9c0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebca00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e9c0b5410 0x7f0e9c0b78d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80ac8e0 tx=0x7f0ea80ec060 comp rx=0 tx=0).stop 2024-09-11T00:23:49.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8104e10 msgr2=0x7f0eb819b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:49.407 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.389+0000 7f0ebca00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8104e10 0x7f0eb819b280 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800eb90 tx=0x7f0ea800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 shutdown_connections 2024-09-11T00:23:49.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e9c0b5410 0x7f0e9c0b78d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0eb810b1a0 0x7f0eb81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0eb810a820 0x7f0eb819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 --2- 172.21.15.103:0/3311329253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0eb8104e10 0x7f0eb819b280 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:49.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 >> 172.21.15.103:0/3311329253 conn(0x7f0eb8100840 msgr2=0x7f0eb8106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:49.408 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 shutdown_connections 2024-09-11T00:23:49.409 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:49.390+0000 7f0ebca00640 1 -- 172.21.15.103:0/3311329253 wait complete. 2024-09-11T00:23:50.286 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T00:23:50.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:50 smithi103 ceph-mon[30578]: pgmap v421: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-11T00:23:50.688 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:23:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:50 smithi183 ceph-mon[35934]: pgmap v421: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-11T00:23:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:50 smithi117 ceph-mon[35525]: pgmap v421: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-11T00:23:51.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 -- 172.21.15.103:0/3617633395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108610 msgr2=0x7feb00108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.336 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 --2- 172.21.15.103:0/3617633395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108610 0x7feb00108a50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feaec009860 tx=0x7feaec02a340 comp rx=0 tx=0).stop 2024-09-11T00:23:51.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 -- 172.21.15.103:0/3617633395 shutdown_connections 2024-09-11T00:23:51.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 --2- 172.21.15.103:0/3617633395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb00108f90 0x7feb0010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 --2- 172.21.15.103:0/3617633395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108610 0x7feb00108a50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 --2- 172.21.15.103:0/3617633395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00102c00 0x7feb00102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 -- 172.21.15.103:0/3617633395 >> 172.21.15.103:0/3617633395 conn(0x7feb000fe6d0 msgr2=0x7feb00100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:51.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 -- 172.21.15.103:0/3617633395 shutdown_connections 2024-09-11T00:23:51.337 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.334+0000 7feb05c00640 1 -- 172.21.15.103:0/3617633395 wait complete. 2024-09-11T00:23:51.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 Processor -- start 2024-09-11T00:23:51.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 -- start start 2024-09-11T00:23:51.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb00102c00 0x7feb0019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00108610 0x7feb0019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108f90 0x7feb001a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb00112c50 con 0x7feb00102c00 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feb00112ad0 con 0x7feb00108f90 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.335+0000 7feb05c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb00112dd0 con 0x7feb00108610 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00108610 0x7feb0019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00108610 0x7feb0019b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39374/0 (socket says 172.21.15.103:39374) 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 -- 172.21.15.103:0/4177334674 learned_addr learned my addr 172.21.15.103:0/4177334674 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaffe00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb00102c00 0x7feb0019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feb05200640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108f90 0x7feb001a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 -- 172.21.15.103:0/4177334674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108f90 msgr2=0x7feb001a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108f90 0x7feb001a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 -- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb00102c00 msgr2=0x7feb0019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.339 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb00102c00 0x7feb0019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feaff400640 1 -- 172.21.15.103:0/4177334674 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb001a1330 con 0x7feb00108610 2024-09-11T00:23:51.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.336+0000 7feb05200640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108f90 0x7feb001a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:23:51.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.337+0000 7feaff400640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00108610 0x7feb0019b7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feaec035040 tx=0x7feaec032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:51.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.337+0000 7feafcc00640 1 -- 172.21.15.103:0/4177334674 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec037040 con 0x7feb00108610 2024-09-11T00:23:51.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.337+0000 7feafcc00640 1 -- 172.21.15.103:0/4177334674 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaec041070 con 0x7feb00108610 2024-09-11T00:23:51.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.337+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb001a15c0 con 0x7feb00108610 2024-09-11T00:23:51.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.337+0000 7feafcc00640 1 -- 172.21.15.103:0/4177334674 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec003850 con 0x7feb00108610 2024-09-11T00:23:51.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.337+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001a7ce0 con 0x7feb00108610 2024-09-11T00:23:51.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.340+0000 7feb09cce740 1 Processor -- start 2024-09-11T00:23:51.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.340+0000 7feb09cce740 1 -- start start 2024-09-11T00:23:51.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feb09cce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feb09cce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feb09cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e120a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feb09cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b025d7ad50 con 0x55b025e0fcb0 2024-09-11T00:23:51.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feb09cce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b025d6a180 con 0x55b025e03400 2024-09-11T00:23:51.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feb09cce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b025d6df90 con 0x55b025e03d20 2024-09-11T00:23:51.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feaffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feb05200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feaffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39376/0 (socket says 172.21.15.103:39376) 2024-09-11T00:23:51.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feaffe00640 1 -- 172.21.15.103:0/2070168984 learned_addr learned my addr 172.21.15.103:0/2070168984 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:51.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.341+0000 7feafcc00640 1 -- 172.21.15.103:0/4177334674 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7feaec0039f0 con 0x7feb00108610 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaffe00640 1 -- 172.21.15.103:0/2070168984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 msgr2=0x55b025e037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaffe00640 1 --2- 172.21.15.103:0/2070168984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaffe00640 1 -- 172.21.15.103:0/2070168984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 msgr2=0x55b025e120a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaffe00640 1 --2- 172.21.15.103:0/2070168984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e120a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaffe00640 1 -- 172.21.15.103:0/2070168984 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b025d34a70 con 0x55b025e03d20 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaffe00640 1 --2- 172.21.15.103:0/2070168984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0f770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009d50 tx=0x7feaf0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf3e0204a31e7d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaf5600640 1 -- 172.21.15.103:0/2070168984 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001e070 con 0x55b025e03d20 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaf5600640 1 -- 172.21.15.103:0/2070168984 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf0019ae0 con 0x55b025e03d20 2024-09-11T00:23:51.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.342+0000 7feaf5600640 1 -- 172.21.15.103:0/2070168984 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf000ea80 con 0x55b025e03d20 2024-09-11T00:23:51.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feb09cce740 1 -- 172.21.15.103:0/2070168984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 msgr2=0x55b025e0f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feb09cce740 1 --2- 172.21.15.103:0/2070168984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0f770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009d50 tx=0x7feaf0011810 comp rx=0 tx=0).stop 2024-09-11T00:23:51.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feb09cce740 1 -- 172.21.15.103:0/2070168984 shutdown_connections 2024-09-11T00:23:51.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feb09cce740 1 --2- 172.21.15.103:0/2070168984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e120a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feb09cce740 1 --2- 172.21.15.103:0/2070168984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0f770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feb09cce740 1 --2- 172.21.15.103:0/2070168984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feb09cce740 1 -- 172.21.15.103:0/2070168984 >> 172.21.15.103:0/2070168984 conn(0x55b025e01230 msgr2=0x55b025e023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.343+0000 7feafcc00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feae40b5410 0x7feae40b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.344+0000 7feafcc00640 1 -- 172.21.15.103:0/4177334674 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feaec1059d0 con 0x7feb00108610 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.344+0000 7feb09cce740 1 -- 172.21.15.103:0/2070168984 shutdown_connections 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.344+0000 7feb05200640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feae40b5410 0x7feae40b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.344+0000 7feb09cce740 1 -- 172.21.15.103:0/2070168984 wait complete. 2024-09-11T00:23:51.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 -- start start 2024-09-11T00:23:51.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb05200640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feae40b5410 0x7feae40b78d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005fd0 tx=0x7feaf8011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:51.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 Processor -- start 2024-09-11T00:23:51.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e0cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e0d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b025bcb5f0 con 0x55b025e0fcb0 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b025d05460 con 0x55b025e03400 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b025e12a50 con 0x55b025e03d20 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.345+0000 7feb09cce740 10 client.0 _ll_register_callbacks cb 0x55b025d15210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e0cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e0cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:60240/0 (socket says 172.21.15.103:60240) 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 -- 172.21.15.103:0/4003440659 learned_addr learned my addr 172.21.15.103:0/4003440659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feb05200640 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e0d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaff400640 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 -- 172.21.15.103:0/4003440659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 msgr2=0x55b025e0d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 -- 172.21.15.103:0/4003440659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 msgr2=0x55b025e0d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e0d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 -- 172.21.15.103:0/4003440659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b025ac8540 con 0x55b025e03400 2024-09-11T00:23:51.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaffe00640 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e0cd10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005890 tx=0x7feaf00056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89935379284f64a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:51.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.346+0000 7feaea000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:23:51.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.347+0000 7feaea000640 1 -- 172.21.15.103:0/4003440659 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001e070 con 0x55b025e03400 2024-09-11T00:23:51.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.347+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b025d07a70 con 0x55b025e03400 2024-09-11T00:23:51.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.347+0000 7feaea000640 1 -- 172.21.15.103:0/4003440659 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feaf000ea00 con 0x55b025e03400 2024-09-11T00:23:51.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.347+0000 7feaea000640 1 -- 172.21.15.103:0/4003440659 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feaf0031050 con 0x55b025e03400 2024-09-11T00:23:51.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.347+0000 7feb09cce740 10 client.25376 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:51.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.347+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b025d226f0 con 0x55b025e03400 2024-09-11T00:23:51.351 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feaea000640 1 -- 172.21.15.103:0/4003440659 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b025d07a70 con 0x55b025e03400 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feaea000640 1 -- 172.21.15.103:0/4003440659 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feaf0023440 con 0x55b025e03400 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feb09cce740 10 client.25376 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feb09cce740 10 client.25376 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feb09cce740 20 client.25376 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feb09cce740 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e20850 0x55b025e21cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feb09cce740 4 client.25376 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b025d7cca0 con 0x55b025e20850 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.348+0000 7feaff400640 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e20850 0x55b025e21cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.349+0000 7feaff400640 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e20850 0x55b025e21cc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=123323c36e15b555 server_cookie=6301f6f09ff020d2 in_seq=0 out_seq=0 2024-09-11T00:23:51.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.349+0000 7feaea000640 10 client.25376 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feaea000640 1 -- 172.21.15.103:0/4003440659 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b025d7cca0 con 0x55b025e20850 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feaea000640 10 client.25376 handle_command_reply: tid=0 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 client.25376 shutdown 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 msgr2=0x55b025e0cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e0cd10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005890 tx=0x7feaf00056f0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 shutdown_connections 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e0d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e20850 0x55b025e21cc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 --2- 172.21.15.103:0/4003440659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03400 0x55b025e0cd10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feaea000640 0 client.25376 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:51.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 >> 172.21.15.103:0/4003440659 conn(0x55b025e01230 msgr2=0x55b025e06dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:51.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.350+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 shutdown_connections 2024-09-11T00:23:51.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.351+0000 7feb09cce740 1 -- 172.21.15.103:0/4003440659 wait complete. 2024-09-11T00:23:51.355 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.351+0000 7feb09cce740 20 client.25376 trim_cache size 0 max 16384 2024-09-11T00:23:51.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 Processor -- start 2024-09-11T00:23:51.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 -- start start 2024-09-11T00:23:51.378 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e03400 0x55b025e02990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e0fcb0 0x55b025e07470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b025e20000 con 0x55b025e03400 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b025e206d0 con 0x55b025e0fcb0 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feb09cce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b025e20da0 con 0x55b025e03d20 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feaffe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e03400 0x55b025e02990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feaffe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e03400 0x55b025e02990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55206/0 (socket says 172.21.15.103:55206) 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.376+0000 7feaffe00640 1 -- 172.21.15.103:0/1644060200 learned_addr learned my addr 172.21.15.103:0/1644060200 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feb05200640 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e0fcb0 0x55b025e07470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feaff400640 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feaffe00640 1 -- 172.21.15.103:0/1644060200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 msgr2=0x55b025e0dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feaffe00640 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feaffe00640 1 -- 172.21.15.103:0/1644060200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e0fcb0 msgr2=0x55b025e07470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feaffe00640 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e0fcb0 0x55b025e07470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feaffe00640 1 -- 172.21.15.103:0/1644060200 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b025e124d0 con 0x55b025e03400 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.377+0000 7feaffe00640 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e03400 0x55b025e02990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009ab0 tx=0x7feaf0009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213bd6943a1a682b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.378+0000 7feaf4c00640 1 -- 172.21.15.103:0/1644060200 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf003a070 con 0x55b025e03400 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.378+0000 7feaf4c00640 1 -- 172.21.15.103:0/1644060200 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf0023e50 con 0x55b025e03400 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.378+0000 7feaf4c00640 1 -- 172.21.15.103:0/1644060200 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001e050 con 0x55b025e03400 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 -- 172.21.15.103:0/1644060200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e03400 msgr2=0x55b025e02990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e03400 0x55b025e02990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009ab0 tx=0x7feaf0009620 comp rx=0 tx=0).stop 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 -- 172.21.15.103:0/1644060200 shutdown_connections 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e0fcb0 0x55b025e07470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.381 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03d20 0x55b025e0dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 --2- 172.21.15.103:0/1644060200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e03400 0x55b025e02990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 -- 172.21.15.103:0/1644060200 >> 172.21.15.103:0/1644060200 conn(0x55b025e01230 msgr2=0x55b025e02280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:51.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 -- 172.21.15.103:0/1644060200 shutdown_connections 2024-09-11T00:23:51.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 -- 172.21.15.103:0/1644060200 wait complete. 2024-09-11T00:23:51.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 -- start start 2024-09-11T00:23:51.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.379+0000 7feb09cce740 1 Processor -- start 2024-09-11T00:23:51.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb09cce740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03400 0x55b025e24270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb09cce740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03d20 0x55b025e247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb09cce740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb09cce740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b025e21b40 con 0x55b025e0fcb0 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb09cce740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b025e21470 con 0x55b025e03d20 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb09cce740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b025e22210 con 0x55b025e03400 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb09cce740 10 client.0 _ll_register_callbacks cb 0x7feaf0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feaffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03400 0x55b025e24270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feb05200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feaffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03400 0x55b025e24270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39406/0 (socket says 172.21.15.103:39406) 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feaffe00640 1 -- 172.21.15.103:0/3329794214 learned_addr learned my addr 172.21.15.103:0/3329794214 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.380+0000 7feaff400640 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03d20 0x55b025e247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaffe00640 1 -- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03d20 msgr2=0x55b025e247b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaffe00640 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03d20 0x55b025e247b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaffe00640 1 -- 172.21.15.103:0/3329794214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 msgr2=0x55b025e299b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:51.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaffe00640 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e299b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaffe00640 1 -- 172.21.15.103:0/3329794214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b025e29ef0 con 0x55b025e03400 2024-09-11T00:23:51.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaff400640 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03d20 0x55b025e247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:51.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaffe00640 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03400 0x55b025e24270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009d50 tx=0x7feaf0023650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=863f64f03efc529f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:51.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaea000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:23:51.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.381+0000 7feaea000640 1 -- 172.21.15.103:0/3329794214 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf003a030 con 0x55b025e03400 2024-09-11T00:23:51.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feaea000640 1 -- 172.21.15.103:0/3329794214 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7feaf000e6e0 con 0x55b025e03400 2024-09-11T00:23:51.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaf0023e50 con 0x55b025e03400 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feaea000640 1 -- 172.21.15.103:0/3329794214 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feaf001e040 con 0x55b025e03400 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feb09cce740 10 client.25408 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b025e8d820 con 0x55b025e03400 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feaea000640 1 -- 172.21.15.103:0/3329794214 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf0023e50 con 0x55b025e03400 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feaea000640 1 -- 172.21.15.103:0/3329794214 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7feaf0003a00 con 0x55b025e03400 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feb09cce740 10 client.25408 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feb09cce740 10 client.25408 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T00:23:51.387 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feb09cce740 20 client.25408 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:51.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.382+0000 7feb09cce740 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e92030 0x55b025e90d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:51.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.383+0000 7feb09cce740 4 client.25408 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T00:23:51.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.383+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "status"}) -- 0x55b025d78750 con 0x55b025e92030 2024-09-11T00:23:51.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.383+0000 7feaff400640 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e92030 0x55b025e90d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:51.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.383+0000 7feaff400640 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e92030 0x55b025e90d40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dae4d6805ff5aa1 server_cookie=711a23b011d31d22 in_seq=0 out_seq=0 2024-09-11T00:23:51.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.383+0000 7feaea000640 10 client.25408 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:51.388 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feaea000640 1 -- 172.21.15.103:0/3329794214 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b025d78750 con 0x55b025e92030 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feaea000640 10 client.25408 handle_command_reply: tid=0 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 client.25408 shutdown 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03400 msgr2=0x55b025e24270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03400 0x55b025e24270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009d50 tx=0x7feaf0023650 comp rx=0 tx=0).stop 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 shutdown_connections 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b025e0fcb0 0x55b025e299b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b025e03d20 0x55b025e247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b025e92030 0x55b025e90d40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 --2- 172.21.15.103:0/3329794214 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b025e03400 0x55b025e24270 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 >> 172.21.15.103:0/3329794214 conn(0x55b025e01230 msgr2=0x55b025e1b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feaea000640 0 client.25408 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 shutdown_connections 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 1 -- 172.21.15.103:0/3329794214 wait complete. 2024-09-11T00:23:51.389 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.384+0000 7feb09cce740 20 client.25408 trim_cache size 0 max 16384 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "cluster_fsid": "40f10072-6fd2-11ef-bcea-c7b262605968", 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "whoami": 0, 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 24656, 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "want_state": "up:active", 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "state": "up:active", 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs", 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "rank_uptime": 202.33016047300001, 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "mdsmap_epoch": 27, 2024-09-11T00:23:51.390 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch": 89, 2024-09-11T00:23:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_epoch_barrier": 89, 2024-09-11T00:23:51.391 INFO:teuthology.orchestra.run.smithi103.stdout: "uptime": 217.30624249100001 2024-09-11T00:23:51.391 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:23:51.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feae40b5410 msgr2=0x7feae40b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feae40b5410 0x7feae40b78d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005fd0 tx=0x7feaf8011040 comp rx=0 tx=0).stop 2024-09-11T00:23:51.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00108610 msgr2=0x7feb0019b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:51.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00108610 0x7feb0019b7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feaec035040 tx=0x7feaec032d50 comp rx=0 tx=0).stop 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 shutdown_connections 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feae40b5410 0x7feae40b78d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb00108f90 0x7feb001a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb00108610 0x7feb0019b7c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 --2- 172.21.15.103:0/4177334674 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb00102c00 0x7feb0019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 >> 172.21.15.103:0/4177334674 conn(0x7feb000fe6d0 msgr2=0x7feb00104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 shutdown_connections 2024-09-11T00:23:51.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:23:51.385+0000 7feb05c00640 1 -- 172.21.15.103:0/4177334674 wait complete. 2024-09-11T00:23:52.390 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T00:23:52.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:52 smithi103 ceph-mon[30578]: pgmap v422: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T00:23:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:52 smithi183 ceph-mon[35934]: pgmap v422: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T00:23:52.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:52 smithi117 ceph-mon[35525]: pgmap v422: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-11T00:23:54.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:54 smithi183 ceph-mon[35934]: pgmap v423: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-11T00:23:54.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:54 smithi103 ceph-mon[30578]: pgmap v423: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-11T00:23:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:54 smithi117 ceph-mon[35525]: pgmap v423: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-09-11T00:23:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 -- 172.21.15.103:0/2672128884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41084a0 msgr2=0x7fccd410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 --2- 172.21.15.103:0/2672128884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41084a0 0x7fccd410a960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009be0 tx=0x7fcccc02a6d0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 -- 172.21.15.103:0/2672128884 shutdown_connections 2024-09-11T00:23:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 --2- 172.21.15.103:0/2672128884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41084a0 0x7fccd410a960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 --2- 172.21.15.103:0/2672128884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd41058f0 0x7fccd4107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 --2- 172.21.15.103:0/2672128884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 0x7fccd4105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 -- 172.21.15.103:0/2672128884 >> 172.21.15.103:0/2672128884 conn(0x7fccd40fcf30 msgr2=0x7fccd40ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 -- 172.21.15.103:0/2672128884 shutdown_connections 2024-09-11T00:23:55.900 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.897+0000 7fccdac00640 1 -- 172.21.15.103:0/2672128884 wait complete. 2024-09-11T00:23:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 Processor -- start 2024-09-11T00:23:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 -- start start 2024-09-11T00:23:55.902 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 0x7fccd419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41058f0 0x7fccd419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd41084a0 0x7fccd419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410e3f0 con 0x7fccd41084a0 2024-09-11T00:23:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410e270 con 0x7fccd406b470 2024-09-11T00:23:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.898+0000 7fccdac00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410e570 con 0x7fccd41058f0 2024-09-11T00:23:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 0x7fccd419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 0x7fccd419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:55950/0 (socket says 172.21.15.103:55950) 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 -- 172.21.15.103:0/1638530155 learned_addr learned my addr 172.21.15.103:0/1638530155 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd8e00640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41058f0 0x7fccd419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccda200640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd41084a0 0x7fccd419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 -- 172.21.15.103:0/1638530155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41058f0 msgr2=0x7fccd419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41058f0 0x7fccd419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 -- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd41084a0 msgr2=0x7fccd419bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd41084a0 0x7fccd419bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 -- 172.21.15.103:0/1638530155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd419c850 con 0x7fccd406b470 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.899+0000 7fccd9800640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 0x7fccd419ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fccc800e9f0 tx=0x7fccc800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.900+0000 7fccd2000640 1 -- 172.21.15.103:0/1638530155 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800cd80 con 0x7fccd406b470 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.900+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd419cb40 con 0x7fccd406b470 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.900+0000 7fccd2000640 1 -- 172.21.15.103:0/1638530155 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc80054d0 con 0x7fccd406b470 2024-09-11T00:23:55.904 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.900+0000 7fccd2000640 1 -- 172.21.15.103:0/1638530155 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8010690 con 0x7fccd406b470 2024-09-11T00:23:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.900+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41a3210 con 0x7fccd406b470 2024-09-11T00:23:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 Processor -- start 2024-09-11T00:23:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 -- start start 2024-09-11T00:23:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd0180 0x565496cd2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd2ab0 0x565496cd4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565496c3a6b0 con 0x565496cd54f0 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565496c29ae0 con 0x565496cd0180 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.903+0000 7fccdf277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565496c2d8f0 con 0x565496cd2ab0 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd2000640 1 -- 172.21.15.103:0/1638530155 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fccc8004e70 con 0x7fccd406b470 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53158/0 (socket says 172.21.15.103:53158) 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 -- 172.21.15.103:0/3513272805 learned_addr learned my addr 172.21.15.103:0/3513272805 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccda200640 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd0180 0x565496cd2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd9800640 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd2ab0 0x565496cd4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 -- 172.21.15.103:0/3513272805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd2ab0 msgr2=0x565496cd4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd2ab0 0x565496cd4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 -- 172.21.15.103:0/3513272805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd0180 msgr2=0x565496cd2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd0180 0x565496cd2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.904+0000 7fccd8e00640 1 -- 172.21.15.103:0/3513272805 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565496bf6780 con 0x565496cd54f0 2024-09-11T00:23:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.905+0000 7fccd8e00640 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd79b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009cc0 tx=0x7fccc0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3047eb4d423361ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.905+0000 7fccc6a00640 1 -- 172.21.15.103:0/3513272805 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0005220 con 0x565496cd54f0 2024-09-11T00:23:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.905+0000 7fccc6a00640 1 -- 172.21.15.103:0/3513272805 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc0004c50 con 0x565496cd54f0 2024-09-11T00:23:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.905+0000 7fccc6a00640 1 -- 172.21.15.103:0/3513272805 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00197f0 con 0x565496cd54f0 2024-09-11T00:23:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccd2000640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fccbc0b52e0 0x7fccbc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 -- 172.21.15.103:0/3513272805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 msgr2=0x565496cd79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd79b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009cc0 tx=0x7fccc0011810 comp rx=0 tx=0).stop 2024-09-11T00:23:55.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccd2000640 1 -- 172.21.15.103:0/1638530155 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fccc80e63b0 con 0x7fccd406b470 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 -- 172.21.15.103:0/3513272805 shutdown_connections 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd79b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd2ab0 0x565496cd4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 --2- 172.21.15.103:0/3513272805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd0180 0x565496cd2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 -- 172.21.15.103:0/3513272805 >> 172.21.15.103:0/3513272805 conn(0x565496cc2ac0 msgr2=0x565496cc3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 -- 172.21.15.103:0/3513272805 shutdown_connections 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccda200640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fccbc0b52e0 0x7fccbc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.906+0000 7fccdf277740 1 -- 172.21.15.103:0/3513272805 wait complete. 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.907+0000 7fccdf277740 1 -- start start 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.907+0000 7fccda200640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fccbc0b52e0 0x7fccbc0b77a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcccc02abf0 tx=0x7fcccc030990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.907+0000 7fccdf277740 1 Processor -- start 2024-09-11T00:23:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccdf277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd0180 0x565496cd9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccdf277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccdf277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd54f0 0x565496cdab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccdf277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565496b61ab0 con 0x565496cd0180 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccdf277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565496bd0570 con 0x565496cd2ab0 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccdf277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565496cd84d0 con 0x565496cd54f0 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccdf277740 10 client.0 _ll_register_callbacks cb 0x565496bdf100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccd9800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd0180 0x565496cd9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccd8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccda200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd54f0 0x565496cdab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccda200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd54f0 0x565496cdab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:52114/0 (socket says 172.21.15.103:52114) 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccd8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:55968/0 (socket says 172.21.15.103:55968) 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.908+0000 7fccda200640 1 -- 172.21.15.103:0/3155543826 learned_addr learned my addr 172.21.15.103:0/3155543826 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccd8e00640 1 -- 172.21.15.103:0/3155543826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd54f0 msgr2=0x565496cdab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccd8e00640 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd54f0 0x565496cdab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccd8e00640 1 -- 172.21.15.103:0/3155543826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd0180 msgr2=0x565496cd9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccd8e00640 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd0180 0x565496cd9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccd8e00640 1 -- 172.21.15.103:0/3155543826 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56549698b790 con 0x565496cd2ab0 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccd8e00640 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd9f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fccc001b040 tx=0x7fccc0005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf9ffab3a1c1b2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccbb400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccbb400640 1 -- 172.21.15.103:0/3155543826 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0029070 con 0x565496cd2ab0 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc0004c50 con 0x565496cd2ab0 2024-09-11T00:23:55.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.909+0000 7fccbb400640 1 -- 172.21.15.103:0/3155543826 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fccc0019940 con 0x565496cd2ab0 2024-09-11T00:23:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccbb400640 1 -- 172.21.15.103:0/3155543826 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fccc0035050 con 0x565496cd2ab0 2024-09-11T00:23:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 10 client.25394 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565496a04620 con 0x565496cd2ab0 2024-09-11T00:23:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccbb400640 1 -- 172.21.15.103:0/3155543826 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc0004c50 con 0x565496cd2ab0 2024-09-11T00:23:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccbb400640 1 -- 172.21.15.103:0/3155543826 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fccc0023550 con 0x565496cd2ab0 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 10 client.25394 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 10 client.25394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 20 client.25394 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496ce4ee0 0x565496ce72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 4 client.25394 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.910+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565496c3c600 con 0x565496ce4ee0 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.911+0000 7fccd9800640 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496ce4ee0 0x565496ce72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.911+0000 7fccd9800640 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496ce4ee0 0x565496ce72d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7df3c290275960 server_cookie=da370b3d3ead289a in_seq=0 out_seq=0 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.911+0000 7fccbb400640 10 client.25394 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccbb400640 1 -- 172.21.15.103:0/3155543826 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565496c3c600 con 0x565496ce4ee0 2024-09-11T00:23:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccbb400640 10 client.25394 handle_command_reply: tid=0 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 client.25394 shutdown 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 msgr2=0x565496cd9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd9f30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fccc001b040 tx=0x7fccc0005e80 comp rx=0 tx=0).stop 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 shutdown_connections 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496ce4ee0 0x565496ce72d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd54f0 0x565496cdab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd9f30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccbb400640 0 client.25394 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 --2- 172.21.15.103:0/3155543826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd0180 0x565496cd9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.912+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 >> 172.21.15.103:0/3155543826 conn(0x565496cc2ac0 msgr2=0x565496cc1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.913+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 shutdown_connections 2024-09-11T00:23:55.918 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.913+0000 7fccdf277740 1 -- 172.21.15.103:0/3155543826 wait complete. 2024-09-11T00:23:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.913+0000 7fccdf277740 20 client.25394 trim_cache size 0 max 16384 2024-09-11T00:23:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 Processor -- start 2024-09-11T00:23:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 -- start start 2024-09-11T00:23:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496cbdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41abb20 con 0x565496cd54f0 2024-09-11T00:23:55.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41a93f0 con 0x565496cd2ab0 2024-09-11T00:23:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccdf277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41a7260 con 0x565496cd0180 2024-09-11T00:23:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccda200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccda200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53190/0 (socket says 172.21.15.103:53190) 2024-09-11T00:23:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccd9800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496cbdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccd8e00640 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.928+0000 7fccda200640 1 -- 172.21.15.103:0/693329239 learned_addr learned my addr 172.21.15.103:0/693329239 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccd8e00640 1 -- 172.21.15.103:0/693329239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 msgr2=0x565496cbdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccd8e00640 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496cbdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccd8e00640 1 -- 172.21.15.103:0/693329239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 msgr2=0x565496cd5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccd8e00640 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccd8e00640 1 -- 172.21.15.103:0/693329239 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565496ce7640 con 0x565496cd2ab0 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccd8e00640 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd58d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fccc0019500 tx=0x7fccc0009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32178cb6880d8da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccc6000640 1 -- 172.21.15.103:0/693329239 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0029070 con 0x565496cd2ab0 2024-09-11T00:23:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccc6000640 1 -- 172.21.15.103:0/693329239 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc0005620 con 0x565496cd2ab0 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.929+0000 7fccc6000640 1 -- 172.21.15.103:0/693329239 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0004e10 con 0x565496cd2ab0 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 -- 172.21.15.103:0/693329239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 msgr2=0x565496cd58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd58d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fccc0019500 tx=0x7fccc0009640 comp rx=0 tx=0).stop 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 -- 172.21.15.103:0/693329239 shutdown_connections 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496cd5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496cd58d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 --2- 172.21.15.103:0/693329239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496cbdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 -- 172.21.15.103:0/693329239 >> 172.21.15.103:0/693329239 conn(0x565496cc2ac0 msgr2=0x565496cc6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 -- 172.21.15.103:0/693329239 shutdown_connections 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 -- 172.21.15.103:0/693329239 wait complete. 2024-09-11T00:23:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.930+0000 7fccdf277740 1 -- start start 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 1 Processor -- start 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496ce1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496ce0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496ce0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41a78b0 con 0x565496cd54f0 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41a8d20 con 0x565496cd2ab0 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41a7f80 con 0x565496cd0180 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccdf277740 10 client.0 _ll_register_callbacks cb 0x7fccc0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccda200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496ce0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccda200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496ce0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53202/0 (socket says 172.21.15.103:53202) 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccda200640 1 -- 172.21.15.103:0/617622861 learned_addr learned my addr 172.21.15.103:0/617622861 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccd9800640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496ce1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.936 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.931+0000 7fccd8e00640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496ce0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccda200640 1 -- 172.21.15.103:0/617622861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 msgr2=0x565496ce1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccda200640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496ce1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccda200640 1 -- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 msgr2=0x565496ce0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccda200640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496ce0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccda200640 1 -- 172.21.15.103:0/617622861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565496cdee40 con 0x565496cd54f0 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccd8e00640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496ce0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccda200640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496ce0b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcccc004af0 tx=0x7fcccc02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7019b38f8e87682 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:23:55.937 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccbb400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:23:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.932+0000 7fccbb400640 1 -- 172.21.15.103:0/617622861 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc036040 con 0x565496cd54f0 2024-09-11T00:23:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccbb400640 1 -- 172.21.15.103:0/617622861 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fcccc004450 con 0x565496cd54f0 2024-09-11T00:23:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc0005620 con 0x565496cd54f0 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccbb400640 1 -- 172.21.15.103:0/617622861 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcccc05f050 con 0x565496cd54f0 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 10 client.15387 fetch_fsmap learned FSMap version 27 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x565496d52180 con 0x565496cd54f0 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccbb400640 1 -- 172.21.15.103:0/617622861 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc0005620 con 0x565496cd54f0 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccbb400640 1 -- 172.21.15.103:0/617622861 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fcccc054020 con 0x565496cd54f0 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 10 client.15387 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 20 client.15387 populate_metadata read hostname 'smithi103' 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496d52b60 0x565496d54f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 4 client.15387 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.933+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565496cbe260 con 0x565496d52b60 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.934+0000 7fccd9800640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496d52b60 0x565496d54f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:23:55.939 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.934+0000 7fccbb400640 10 client.15387 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:55.940 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.934+0000 7fccd9800640 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496d52b60 0x565496d54f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d19b1972f4e4c35 server_cookie=55a5232cc2bd9075 in_seq=0 out_seq=0 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.942+0000 7fccbb400640 1 -- 172.21.15.103:0/617622861 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565496cbe260 con 0x565496d52b60 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.942+0000 7fccbb400640 10 client.15387 handle_command_reply: tid=0 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.942+0000 7fccdf277740 1 client.15387 shutdown 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 msgr2=0x565496ce0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496ce0b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcccc004af0 tx=0x7fcccc02a6d0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 shutdown_connections 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565496cd54f0 0x565496ce0b00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565496cd2ab0 0x565496ce0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565496d52b60 0x565496d54f80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 --2- 172.21.15.103:0/617622861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565496cd0180 0x565496ce1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccbb400640 0 client.15387 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 >> 172.21.15.103:0/617622861 conn(0x565496cc2ac0 msgr2=0x565496cd58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 shutdown_connections 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 1 -- 172.21.15.103:0/617622861 wait complete. 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.943+0000 7fccdf277740 20 client.15387 trim_cache size 0 max 16384 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:23:55.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:23:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fccbc0b52e0 msgr2=0x7fccbc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fccbc0b52e0 0x7fccbc0b77a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcccc02abf0 tx=0x7fcccc030990 comp rx=0 tx=0).stop 2024-09-11T00:23:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 msgr2=0x7fccd419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:23:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 0x7fccd419ad10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fccc800e9f0 tx=0x7fccc800edf0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 shutdown_connections 2024-09-11T00:23:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fccbc0b52e0 0x7fccbc0b77a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccd41084a0 0x7fccd419bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fccd41058f0 0x7fccd419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 --2- 172.21.15.103:0/1638530155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccd406b470 0x7fccd419ad10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:23:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 >> 172.21.15.103:0/1638530155 conn(0x7fccd40fcf30 msgr2=0x7fccd40ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:23:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 shutdown_connections 2024-09-11T00:23:55.948 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:23:55.944+0000 7fccdac00640 1 -- 172.21.15.103:0/1638530155 wait complete. 2024-09-11T00:23:56.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:56 smithi183 ceph-mon[35934]: pgmap v424: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-11T00:23:56.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:56 smithi103 ceph-mon[30578]: pgmap v424: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-11T00:23:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:56 smithi117 ceph-mon[35525]: pgmap v424: 129 pgs: 129 active+clean; 659 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-11T00:23:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:23:58 smithi183 ceph-mon[35934]: pgmap v425: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-11T00:23:58.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:23:58 smithi103 ceph-mon[30578]: pgmap v425: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-11T00:23:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:23:58 smithi117 ceph-mon[35525]: pgmap v425: 129 pgs: 129 active+clean; 658 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-11T00:24:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:00 smithi183 ceph-mon[35934]: pgmap v426: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T00:24:00.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:00 smithi103 ceph-mon[30578]: pgmap v426: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T00:24:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:00 smithi117 ceph-mon[35525]: pgmap v426: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-09-11T00:24:02.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:02 smithi183 ceph-mon[35934]: pgmap v427: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:02.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:02 smithi103 ceph-mon[30578]: pgmap v427: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:02 smithi117 ceph-mon[35525]: pgmap v427: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:04 smithi183 ceph-mon[35934]: pgmap v428: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-11T00:24:04.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:04 smithi103 ceph-mon[30578]: pgmap v428: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-11T00:24:04.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:04 smithi117 ceph-mon[35525]: pgmap v428: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-11T00:24:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:06 smithi183 ceph-mon[35934]: pgmap v429: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:06.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:06 smithi103 ceph-mon[30578]: pgmap v429: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:06.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:06 smithi117 ceph-mon[35525]: pgmap v429: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:07.391 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-11T00:24:07.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:24:07.815 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:24:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 -- 172.21.15.103:0/882956067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10a820 msgr2=0x7f6d6c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:08.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 --2- 172.21.15.103:0/882956067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10a820 0x7f6d6c10ac60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009900 tx=0x7f6d5802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 -- 172.21.15.103:0/882956067 shutdown_connections 2024-09-11T00:24:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 --2- 172.21.15.103:0/882956067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10b1a0 0x7f6d6c1119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 --2- 172.21.15.103:0/882956067 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10a820 0x7f6d6c10ac60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 --2- 172.21.15.103:0/882956067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c104e10 0x7f6d6c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 -- 172.21.15.103:0/882956067 >> 172.21.15.103:0/882956067 conn(0x7f6d6c100840 msgr2=0x7f6d6c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 -- 172.21.15.103:0/882956067 shutdown_connections 2024-09-11T00:24:08.511 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.508+0000 7f6d6be00640 1 -- 172.21.15.103:0/882956067 wait complete. 2024-09-11T00:24:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.509+0000 7f6d6be00640 1 Processor -- start 2024-09-11T00:24:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.509+0000 7f6d6be00640 1 -- start start 2024-09-11T00:24:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c104e10 0x7f6d6c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:08.512 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10a820 0x7f6d6c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:08.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10b1a0 0x7f6d6c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:08.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c114e60 con 0x7f6d6c10a820 2024-09-11T00:24:08.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c114ce0 con 0x7f6d6c104e10 2024-09-11T00:24:08.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c114fe0 con 0x7f6d6c10b1a0 2024-09-11T00:24:08.513 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10a820 0x7f6d6c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:08.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10a820 0x7f6d6c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60822/0 (socket says 172.21.15.103:60822) 2024-09-11T00:24:08.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6a000640 1 -- 172.21.15.103:0/4200397298 learned_addr learned my addr 172.21.15.103:0/4200397298 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:08.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.510+0000 7f6d6b400640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10b1a0 0x7f6d6c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:08.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.511+0000 7f6d6aa00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c104e10 0x7f6d6c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:08.514 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.511+0000 7f6d6a000640 1 -- 172.21.15.103:0/4200397298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10b1a0 msgr2=0x7f6d6c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:08.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.511+0000 7f6d6a000640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10b1a0 0x7f6d6c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.511+0000 7f6d6a000640 1 -- 172.21.15.103:0/4200397298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c104e10 msgr2=0x7f6d6c19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:08.515 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.511+0000 7f6d6a000640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c104e10 0x7f6d6c19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.511+0000 7f6d6a000640 1 -- 172.21.15.103:0/4200397298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c1a12e0 con 0x7f6d6c10a820 2024-09-11T00:24:08.516 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.512+0000 7f6d6a000640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10a820 0x7f6d6c19b770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005940 tx=0x7f6d580040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:08.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.512+0000 7f6d63400640 1 -- 172.21.15.103:0/4200397298 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5803f030 con 0x7f6d6c10a820 2024-09-11T00:24:08.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.512+0000 7f6d63400640 1 -- 172.21.15.103:0/4200397298 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d580044c0 con 0x7f6d6c10a820 2024-09-11T00:24:08.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.512+0000 7f6d63400640 1 -- 172.21.15.103:0/4200397298 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58035070 con 0x7f6d6c10a820 2024-09-11T00:24:08.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.512+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6c1a1570 con 0x7f6d6c10a820 2024-09-11T00:24:08.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.513+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6c1a7c90 con 0x7f6d6c10a820 2024-09-11T00:24:08.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.514+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6c1051f0 con 0x7f6d6c10a820 2024-09-11T00:24:08.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.518+0000 7f6d63400640 1 -- 172.21.15.103:0/4200397298 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6d5804c070 con 0x7f6d6c10a820 2024-09-11T00:24:08.522 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.521+0000 7f6d63400640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d500b5490 0x7f6d500b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:08.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.521+0000 7f6d63400640 1 -- 172.21.15.103:0/4200397298 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6d58106810 con 0x7f6d6c10a820 2024-09-11T00:24:08.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.521+0000 7f6d6aa00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d500b5490 0x7f6d500b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:08.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.522+0000 7f6d6aa00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d500b5490 0x7f6d500b7950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009bd0 tx=0x7f6d5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:08.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.524+0000 7f6d63400640 1 -- 172.21.15.103:0/4200397298 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6d58049020 con 0x7f6d6c10a820 2024-09-11T00:24:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:08 smithi103 ceph-mon[30578]: pgmap v430: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:08.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.753+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d6c1a7ec0 con 0x7f6d6c10a820 2024-09-11T00:24:08.756 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:24:08.756 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-11T00:23:29:477265+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:23:28.708651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:24:08.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.754+0000 7f6d63400640 1 -- 172.21.15.103:0/4200397298 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11933 (secure 0 0 0) 0x7f6d580cd2e0 con 0x7f6d6c10a820 2024-09-11T00:24:08.757 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-11T00:24:08.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d500b5490 msgr2=0x7f6d500b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:08.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d500b5490 0x7f6d500b7950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009bd0 tx=0x7f6d5c005f90 comp rx=0 tx=0).stop 2024-09-11T00:24:08.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10a820 msgr2=0x7f6d6c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:08.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10a820 0x7f6d6c19b770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6d58005940 tx=0x7f6d580040c0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 shutdown_connections 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d500b5490 0x7f6d500b7950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d6c10b1a0 0x7f6d6c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d6c10a820 0x7f6d6c19b770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 --2- 172.21.15.103:0/4200397298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d6c104e10 0x7f6d6c19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.757+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 >> 172.21.15.103:0/4200397298 conn(0x7f6d6c100840 msgr2=0x7f6d6c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.758+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 shutdown_connections 2024-09-11T00:24:08.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:08.758+0000 7f6d6be00640 1 -- 172.21.15.103:0/4200397298 wait complete. 2024-09-11T00:24:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:08 smithi183 ceph-mon[35934]: pgmap v430: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:08 smithi117 ceph-mon[35525]: pgmap v430: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 -- 172.21.15.103:0/1440300254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 msgr2=0x7fd77c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 --2- 172.21.15.103:0/1440300254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 0x7fd77c105b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd764009900 tx=0x7fd76402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 -- 172.21.15.103:0/1440300254 shutdown_connections 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 --2- 172.21.15.103:0/1440300254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c106290 0x7fd77c106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 --2- 172.21.15.103:0/1440300254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 0x7fd77c105b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 --2- 172.21.15.103:0/1440300254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c06b470 0x7fd77c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 -- 172.21.15.103:0/1440300254 >> 172.21.15.103:0/1440300254 conn(0x7fd77c0fcf30 msgr2=0x7fd77c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.102+0000 7fd77be00640 1 -- 172.21.15.103:0/1440300254 shutdown_connections 2024-09-11T00:24:09.105 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.103+0000 7fd77be00640 1 -- 172.21.15.103:0/1440300254 wait complete. 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 Processor -- start 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 -- start start 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c06b470 0x7fd77c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 0x7fd77c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c106290 0x7fd77c1a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c1149d0 con 0x7fd77c06b470 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c114850 con 0x7fd77c106290 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.104+0000 7fd77be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd77c114b50 con 0x7fd77c1036e0 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c06b470 0x7fd77c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c06b470 0x7fd77c19f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60844/0 (socket says 172.21.15.103:60844) 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 -- 172.21.15.103:0/4182495550 learned_addr learned my addr 172.21.15.103:0/4182495550 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd773e00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 0x7fd77c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77b400640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c106290 0x7fd77c1a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 -- 172.21.15.103:0/4182495550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 msgr2=0x7fd77c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 0x7fd77c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 -- 172.21.15.103:0/4182495550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c106290 msgr2=0x7fd77c1a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c106290 0x7fd77c1a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77aa00640 1 -- 172.21.15.103:0/4182495550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77c1a0c70 con 0x7fd77c06b470 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd77b400640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c106290 0x7fd77c1a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.105+0000 7fd773e00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 0x7fd77c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.106+0000 7fd77aa00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c06b470 0x7fd77c19f130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d950 tx=0x7fd76c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.106+0000 7fd773400640 1 -- 172.21.15.103:0/4182495550 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c017070 con 0x7fd77c06b470 2024-09-11T00:24:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.106+0000 7fd773400640 1 -- 172.21.15.103:0/4182495550 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd76c009c30 con 0x7fd77c06b470 2024-09-11T00:24:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.106+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77c1a0f60 con 0x7fd77c06b470 2024-09-11T00:24:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.106+0000 7fd773400640 1 -- 172.21.15.103:0/4182495550 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c00f620 con 0x7fd77c06b470 2024-09-11T00:24:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.106+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd77c1a7630 con 0x7fd77c06b470 2024-09-11T00:24:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.110+0000 7fd7843ec740 1 Processor -- start 2024-09-11T00:24:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.110+0000 7fd7843ec740 1 -- start start 2024-09-11T00:24:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c912d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c915690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c9181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd7843ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b4c8836b0 con 0x559b4c9132a0 2024-09-11T00:24:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd7843ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b4c872ae0 con 0x559b4c915ce0 2024-09-11T00:24:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd7843ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b4c8768f0 con 0x559b4c910970 2024-09-11T00:24:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd773400640 1 -- 172.21.15.103:0/4182495550 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd76c00f7c0 con 0x7fd77c06b470 2024-09-11T00:24:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd773e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c9181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd773e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c9181a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45098/0 (socket says 172.21.15.103:45098) 2024-09-11T00:24:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd773e00640 1 -- 172.21.15.103:0/2092422631 learned_addr learned my addr 172.21.15.103:0/2092422631 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.111+0000 7fd77b400640 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c912d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd77aa00640 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c915690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd773e00640 1 -- 172.21.15.103:0/2092422631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 msgr2=0x559b4c912d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd773e00640 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c912d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd773e00640 1 -- 172.21.15.103:0/2092422631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 msgr2=0x559b4c915690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd773e00640 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c915690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd773e00640 1 -- 172.21.15.103:0/2092422631 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4c83f780 con 0x559b4c915ce0 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd773e00640 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c9181a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd764009900 tx=0x7fd7640315a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9ed044887f093a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.112+0000 7fd76b400640 1 -- 172.21.15.103:0/2092422631 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764040070 con 0x559b4c915ce0 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.113+0000 7fd76b400640 1 -- 172.21.15.103:0/2092422631 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd764005040 con 0x559b4c915ce0 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.113+0000 7fd76b400640 1 -- 172.21.15.103:0/2092422631 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764035040 con 0x559b4c915ce0 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.113+0000 7fd773400640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd7600b5430 0x7fd7600b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.113+0000 7fd7843ec740 1 -- 172.21.15.103:0/2092422631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 msgr2=0x559b4c9181a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.113+0000 7fd7843ec740 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c9181a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd764009900 tx=0x7fd7640315a0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.113+0000 7fd773400640 1 -- 172.21.15.103:0/4182495550 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd76c0e59a0 con 0x7fd77c06b470 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd7843ec740 1 -- 172.21.15.103:0/2092422631 shutdown_connections 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd7843ec740 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c9181a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd7843ec740 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c915690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd7843ec740 1 --2- 172.21.15.103:0/2092422631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c912d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd7843ec740 1 -- 172.21.15.103:0/2092422631 >> 172.21.15.103:0/2092422631 conn(0x559b4c90bac0 msgr2=0x559b4c90c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd77b400640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd7600b5430 0x7fd7600b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd7843ec740 1 -- 172.21.15.103:0/2092422631 shutdown_connections 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.114+0000 7fd7843ec740 1 -- 172.21.15.103:0/2092422631 wait complete. 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.115+0000 7fd7843ec740 1 -- start start 2024-09-11T00:24:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.115+0000 7fd77b400640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd7600b5430 0x7fd7600b78f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd774009fd0 tx=0x7fd774009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.115+0000 7fd7843ec740 1 Processor -- start 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.115+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c910970 0x559b4c91a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.115+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c91a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.115+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c915ce0 0x559b4c91b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd7843ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b4c7aaab0 con 0x559b4c9132a0 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd7843ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559b4c819570 con 0x559b4c910970 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd7843ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559b4c918c60 con 0x559b4c915ce0 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd7843ec740 10 client.0 _ll_register_callbacks cb 0x559b4c828100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd773e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c91a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c910970 0x559b4c91a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c910970 0x559b4c91a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45102/0 (socket says 172.21.15.103:45102) 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 -- 172.21.15.103:0/1998994371 learned_addr learned my addr 172.21.15.103:0/1998994371 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd773e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c91a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60864/0 (socket says 172.21.15.103:60864) 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77b400640 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c915ce0 0x559b4c91b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 -- 172.21.15.103:0/1998994371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c915ce0 msgr2=0x559b4c91b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c915ce0 0x559b4c91b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 -- 172.21.15.103:0/1998994371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 msgr2=0x559b4c91a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c91a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.116+0000 7fd77aa00640 1 -- 172.21.15.103:0/1998994371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4c5d4790 con 0x559b4c910970 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd77aa00640 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c910970 0x559b4c91a100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0abe10 tx=0x7fd76c0ab7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a0dd3cbfb61ef87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd75fe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd75fe00640 1 -- 172.21.15.103:0/1998994371 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c020040 con 0x559b4c910970 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd75fe00640 1 -- 172.21.15.103:0/1998994371 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd76c0ab0e0 con 0x559b4c910970 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd764005040 con 0x559b4c910970 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd75fe00640 1 -- 172.21.15.103:0/1998994371 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd76c010460 con 0x559b4c910970 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd7843ec740 10 client.25418 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b4c64d620 con 0x559b4c910970 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.117+0000 7fd75fe00640 1 -- 172.21.15.103:0/1998994371 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd764005040 con 0x559b4c910970 2024-09-11T00:24:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd75fe00640 1 -- 172.21.15.103:0/1998994371 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd76c0aab50 con 0x559b4c910970 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd7843ec740 10 client.25418 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd7843ec740 10 client.25418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd7843ec740 20 client.25418 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd7843ec740 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c925a50 0x559b4c927ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd7843ec740 4 client.25418 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b4c885600 con 0x559b4c925a50 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.118+0000 7fd773e00640 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c925a50 0x559b4c927ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.119+0000 7fd773e00640 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c925a50 0x559b4c927ec0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5c2bc2f109929ce server_cookie=d3ee26f3a0c946c3 in_seq=0 out_seq=0 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.119+0000 7fd75fe00640 10 client.25418 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.119+0000 7fd75fe00640 1 -- 172.21.15.103:0/1998994371 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559b4c885600 con 0x559b4c925a50 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.119+0000 7fd75fe00640 10 client.25418 handle_command_reply: tid=0 2024-09-11T00:24:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 client.25418 shutdown 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c910970 msgr2=0x559b4c91a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c910970 0x559b4c91a100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0abe10 tx=0x7fd76c0ab7d0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 shutdown_connections 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c925a50 0x559b4c927ec0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c915ce0 0x559b4c91b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c91a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd75fe00640 0 client.25418 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 --2- 172.21.15.103:0/1998994371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c910970 0x559b4c91a100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 >> 172.21.15.103:0/1998994371 conn(0x559b4c90bac0 msgr2=0x559b4c90abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 shutdown_connections 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 1 -- 172.21.15.103:0/1998994371 wait complete. 2024-09-11T00:24:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.120+0000 7fd7843ec740 20 client.25418 trim_cache size 0 max 16384 2024-09-11T00:24:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.135+0000 7fd7843ec740 1 Processor -- start 2024-09-11T00:24:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.135+0000 7fd7843ec740 1 -- start start 2024-09-11T00:24:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c9160c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c9132a0 0x559b4c916600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c915ce0 0x559b4c90c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd7843ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7580123a0 con 0x559b4c915ce0 2024-09-11T00:24:09.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd7843ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd75800fc70 con 0x559b4c9132a0 2024-09-11T00:24:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd7843ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd75800dac0 con 0x559b4c910970 2024-09-11T00:24:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd77b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c915ce0 0x559b4c90c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd77aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c9160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd77b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c915ce0 0x559b4c90c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60866/0 (socket says 172.21.15.103:60866) 2024-09-11T00:24:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd77b400640 1 -- 172.21.15.103:0/3821892256 learned_addr learned my addr 172.21.15.103:0/3821892256 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.136+0000 7fd773e00640 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c9132a0 0x559b4c916600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd773e00640 1 -- 172.21.15.103:0/3821892256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 msgr2=0x559b4c9160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd773e00640 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c9160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd773e00640 1 -- 172.21.15.103:0/3821892256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c915ce0 msgr2=0x559b4c90c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd773e00640 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c915ce0 0x559b4c90c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd773e00640 1 -- 172.21.15.103:0/3821892256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4c9130d0 con 0x559b4c9132a0 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd77aa00640 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c9160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd773e00640 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c9132a0 0x559b4c916600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd764005ab0 tx=0x7fd76402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e3ef75376d130b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd76aa00640 1 -- 172.21.15.103:0/3821892256 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764035040 con 0x559b4c9132a0 2024-09-11T00:24:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd76aa00640 1 -- 172.21.15.103:0/3821892256 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd764005470 con 0x559b4c9132a0 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.137+0000 7fd76aa00640 1 -- 172.21.15.103:0/3821892256 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764040040 con 0x559b4c9132a0 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 -- 172.21.15.103:0/3821892256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c9132a0 msgr2=0x559b4c916600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c9132a0 0x559b4c916600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd764005ab0 tx=0x7fd76402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 -- 172.21.15.103:0/3821892256 shutdown_connections 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c915ce0 0x559b4c90c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c9132a0 0x559b4c916600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 --2- 172.21.15.103:0/3821892256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c9160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 -- 172.21.15.103:0/3821892256 >> 172.21.15.103:0/3821892256 conn(0x559b4c90bac0 msgr2=0x559b4c90fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 -- 172.21.15.103:0/3821892256 shutdown_connections 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.138+0000 7fd7843ec740 1 -- 172.21.15.103:0/3821892256 wait complete. 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 -- start start 2024-09-11T00:24:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 Processor -- start 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c921c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c920980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c920f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd75800e130 con 0x559b4c9132a0 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd75800f5a0 con 0x559b4c915ce0 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.139+0000 7fd7843ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd75800e800 con 0x559b4c910970 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c921c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd773e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c920980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c921c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39220/0 (socket says 172.21.15.103:39220) 2024-09-11T00:24:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 -- 172.21.15.103:0/478387302 learned_addr learned my addr 172.21.15.103:0/478387302 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77b400640 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c920f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd7843ec740 10 client.0 _ll_register_callbacks cb 0x7fd76c0a7d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 -- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 msgr2=0x559b4c920f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c920f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 -- 172.21.15.103:0/478387302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 msgr2=0x559b4c920980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c920980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.140+0000 7fd77aa00640 1 -- 172.21.15.103:0/478387302 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4c921770 con 0x559b4c910970 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd77aa00640 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c921c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0abb30 tx=0x7fd76c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f035e277027a23ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd75fe00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd75fe00640 1 -- 172.21.15.103:0/478387302 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c014030 con 0x559b4c910970 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd75fe00640 1 -- 172.21.15.103:0/478387302 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd76c0ab2f0 con 0x559b4c910970 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd764005470 con 0x559b4c910970 2024-09-11T00:24:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd75fe00640 1 -- 172.21.15.103:0/478387302 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd76c020070 con 0x559b4c910970 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd7843ec740 10 client.25450 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559b4c992ac0 con 0x559b4c910970 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.141+0000 7fd75fe00640 1 -- 172.21.15.103:0/478387302 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd764005470 con 0x559b4c910970 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd75fe00640 1 -- 172.21.15.103:0/478387302 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fd76c0cd310 con 0x559b4c910970 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd7843ec740 10 client.25450 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd7843ec740 10 client.25450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd7843ec740 20 client.25450 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd7843ec740 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c993290 0x559b4c9956b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd7843ec740 4 client.25450 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b4c8810b0 con 0x559b4c993290 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.142+0000 7fd773e00640 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c993290 0x559b4c9956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.143+0000 7fd773e00640 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c993290 0x559b4c9956b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a428d4f158c7963d server_cookie=d2d438848a478c8c in_seq=0 out_seq=0 2024-09-11T00:24:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.143+0000 7fd75fe00640 10 client.25450 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.149+0000 7fd75fe00640 1 -- 172.21.15.103:0/478387302 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b4c8810b0 con 0x559b4c993290 2024-09-11T00:24:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.149+0000 7fd75fe00640 10 client.25450 handle_command_reply: tid=0 2024-09-11T00:24:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.149+0000 7fd7843ec740 1 client.25450 shutdown 2024-09-11T00:24:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 msgr2=0x559b4c921c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c921c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0abb30 tx=0x7fd76c000bc0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 shutdown_connections 2024-09-11T00:24:09.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559b4c915ce0 0x559b4c920f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b4c9132a0 0x559b4c920980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559b4c993290 0x559b4c9956b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 --2- 172.21.15.103:0/478387302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559b4c910970 0x559b4c921c80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 >> 172.21.15.103:0/478387302 conn(0x559b4c90bac0 msgr2=0x559b4c90c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd75fe00640 0 client.25450 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 shutdown_connections 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 1 -- 172.21.15.103:0/478387302 wait complete. 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.150+0000 7fd7843ec740 20 client.25450 trim_cache size 0 max 16384 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:24:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:24:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:24:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd7600b5430 msgr2=0x7fd7600b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd7600b5430 0x7fd7600b78f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd774009fd0 tx=0x7fd774009500 comp rx=0 tx=0).stop 2024-09-11T00:24:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c06b470 msgr2=0x7fd77c19f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c06b470 0x7fd77c19f130 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00d950 tx=0x7fd76c00de10 comp rx=0 tx=0).stop 2024-09-11T00:24:09.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 shutdown_connections 2024-09-11T00:24:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd7600b5430 0x7fd7600b78f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd77c106290 0x7fd77c1a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd77c1036e0 0x7fd77c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 --2- 172.21.15.103:0/4182495550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd77c06b470 0x7fd77c19f130 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 >> 172.21.15.103:0/4182495550 conn(0x7fd77c0fcf30 msgr2=0x7fd77c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 shutdown_connections 2024-09-11T00:24:09.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:09.151+0000 7fd77be00640 1 -- 172.21.15.103:0/4182495550 wait complete. 2024-09-11T00:24:09.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:09 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4200397298' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:09.801 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:09 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4200397298' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:09.861 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-11T00:24:09.862 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-11T00:24:09.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:09 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4200397298' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:10.358 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:24:10.757 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:10 smithi103 ceph-mon[30578]: pgmap v431: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:24:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:10 smithi183 ceph-mon[35934]: pgmap v431: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:24:10.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:10 smithi117 ceph-mon[35525]: pgmap v431: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.134+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3901214795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58103b10 msgr2=0x7f7f58103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.134+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3901214795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58103b10 0x7f7f58103ef0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009900 tx=0x7f7f4802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.134+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3901214795 shutdown_connections 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.134+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3901214795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f5810cf70 0x7f7f5810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.134+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3901214795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f58104430 0x7f7f5810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.134+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3901214795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58103b10 0x7f7f58103ef0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.134+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3901214795 >> 172.21.15.103:0/3901214795 conn(0x7f7f580fd5b0 msgr2=0x7f7f580ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:11.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.136+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3901214795 shutdown_connections 2024-09-11T00:24:11.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.136+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3901214795 wait complete. 2024-09-11T00:24:11.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.137+0000 7f7f5ca00640 1 Processor -- start 2024-09-11T00:24:11.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.138+0000 7f7f5ca00640 1 -- start start 2024-09-11T00:24:11.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.138+0000 7f7f5ca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f58103b10 0x7f7f5810c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.138+0000 7f7f5ca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58104430 0x7f7f581097e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.138+0000 7f7f5ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f5810cf70 0x7f7f58109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.138+0000 7f7f5ca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581030f0 con 0x7f7f58103b10 2024-09-11T00:24:11.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.138+0000 7f7f5ca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58102f70 con 0x7f7f5810cf70 2024-09-11T00:24:11.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.138+0000 7f7f5ca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58103270 con 0x7f7f58104430 2024-09-11T00:24:11.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.139+0000 7f7f56800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f58103b10 0x7f7f5810c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.139+0000 7f7f56800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f58103b10 0x7f7f5810c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60888/0 (socket says 172.21.15.103:60888) 2024-09-11T00:24:11.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.139+0000 7f7f56800640 1 -- 172.21.15.103:0/3375741818 learned_addr learned my addr 172.21.15.103:0/3375741818 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:11.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.139+0000 7f7f55e00640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58104430 0x7f7f581097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.140+0000 7f7f57200640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f5810cf70 0x7f7f58109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.140+0000 7f7f56800640 1 -- 172.21.15.103:0/3375741818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58104430 msgr2=0x7f7f581097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.142 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.140+0000 7f7f56800640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58104430 0x7f7f581097e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.140+0000 7f7f56800640 1 -- 172.21.15.103:0/3375741818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f5810cf70 msgr2=0x7f7f58109d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.143 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.140+0000 7f7f56800640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f5810cf70 0x7f7f58109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.140+0000 7f7f56800640 1 -- 172.21.15.103:0/3375741818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5810a4f0 con 0x7f7f58103b10 2024-09-11T00:24:11.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.140+0000 7f7f56800640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f58103b10 0x7f7f5810c590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005940 tx=0x7f7f48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:11.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.141+0000 7f7f4f400640 1 -- 172.21.15.103:0/3375741818 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4803f030 con 0x7f7f58103b10 2024-09-11T00:24:11.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.141+0000 7f7f4f400640 1 -- 172.21.15.103:0/3375741818 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f48003aa0 con 0x7f7f58103b10 2024-09-11T00:24:11.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.141+0000 7f7f4f400640 1 -- 172.21.15.103:0/3375741818 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48035070 con 0x7f7f58103b10 2024-09-11T00:24:11.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.141+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f581abb20 con 0x7f7f58103b10 2024-09-11T00:24:11.144 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.141+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f581abf80 con 0x7f7f58103b10 2024-09-11T00:24:11.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.146+0000 7f7f5feb4740 1 Processor -- start 2024-09-11T00:24:11.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.146+0000 7f7f5feb4740 1 -- start start 2024-09-11T00:24:11.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.147+0000 7f7f4f400640 1 -- 172.21.15.103:0/3375741818 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7f48003c50 con 0x7f7f58103b10 2024-09-11T00:24:11.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.147+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.147+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.147+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96faf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.147+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5609e9661ca0 con 0x5609e96f8ba0 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.147+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609e96510d0 con 0x5609e96ecc10 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.147+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5609e9654ee0 con 0x5609e96ec2f0 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.148+0000 7f7f57200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.148+0000 7f7f57200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96ec6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39246/0 (socket says 172.21.15.103:39246) 2024-09-11T00:24:11.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.148+0000 7f7f57200640 1 -- 172.21.15.103:0/946738990 learned_addr learned my addr 172.21.15.103:0/946738990 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:11.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.148+0000 7f7f57200640 1 -- 172.21.15.103:0/946738990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 msgr2=0x5609e96f8660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:24:11.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.148+0000 7f7f57200640 1 --2- 172.21.15.103:0/946738990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f8660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.148+0000 7f7f57200640 1 -- 172.21.15.103:0/946738990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 msgr2=0x5609e96faf90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:24:11.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.148+0000 7f7f57200640 1 --2- 172.21.15.103:0/946738990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96faf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.149+0000 7f7f57200640 1 -- 172.21.15.103:0/946738990 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e9603400 con 0x5609e96ec2f0 2024-09-11T00:24:11.152 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.149+0000 7f7f4f400640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7f3c0b5490 0x7f7f3c0b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.150+0000 7f7f57200640 1 --2- 172.21.15.103:0/946738990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96ec6d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009e30 tx=0x7f7f5000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a737958dc7b542a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:11.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.150+0000 7f7f4f400640 1 -- 172.21.15.103:0/3375741818 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7f48106740 con 0x7f7f58103b10 2024-09-11T00:24:11.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.150+0000 7f7f43400640 1 -- 172.21.15.103:0/946738990 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50011640 con 0x5609e96ec2f0 2024-09-11T00:24:11.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.150+0000 7f7f43400640 1 -- 172.21.15.103:0/946738990 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f50011c90 con 0x5609e96ec2f0 2024-09-11T00:24:11.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.150+0000 7f7f43400640 1 -- 172.21.15.103:0/946738990 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5001bc30 con 0x5609e96ec2f0 2024-09-11T00:24:11.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.150+0000 7f7f56800640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7f3c0b5490 0x7f7f3c0b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 -- 172.21.15.103:0/946738990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 msgr2=0x5609e96ec6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/946738990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96ec6d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7f50009e30 tx=0x7f7f5000f730 comp rx=0 tx=0).stop 2024-09-11T00:24:11.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f56800640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7f3c0b5490 0x7f7f3c0b7950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7f48006010 tx=0x7f7f480c9290 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:11.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 -- 172.21.15.103:0/946738990 shutdown_connections 2024-09-11T00:24:11.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/946738990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96faf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/946738990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/946738990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96ec6d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 -- 172.21.15.103:0/946738990 >> 172.21.15.103:0/946738990 conn(0x5609e96ea120 msgr2=0x5609e96eb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:11.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.152+0000 7f7f5feb4740 1 -- 172.21.15.103:0/946738990 shutdown_connections 2024-09-11T00:24:11.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.153+0000 7f7f5feb4740 1 -- 172.21.15.103:0/946738990 wait complete. 2024-09-11T00:24:11.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.153+0000 7f7f5feb4740 1 -- start start 2024-09-11T00:24:11.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.153+0000 7f7f5feb4740 1 Processor -- start 2024-09-11T00:24:11.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.154+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.154+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.154+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.154+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5609e94b27a0 con 0x5609e96f8ba0 2024-09-11T00:24:11.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.154+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5609e95f6a10 con 0x5609e96ecc10 2024-09-11T00:24:11.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.154+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5609e96fb820 con 0x5609e96ec2f0 2024-09-11T00:24:11.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.154+0000 7f7f5feb4740 10 client.0 _ll_register_callbacks cb 0x5609e9606be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:11.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.155+0000 7f7f57200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.155+0000 7f7f55e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.155+0000 7f7f57200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39248/0 (socket says 172.21.15.103:39248) 2024-09-11T00:24:11.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.155+0000 7f7f57200640 1 -- 172.21.15.103:0/658707059 learned_addr learned my addr 172.21.15.103:0/658707059 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:11.158 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.155+0000 7f7f57200640 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.156+0000 7f7f57200640 1 -- 172.21.15.103:0/658707059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 msgr2=0x5609e96f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.156+0000 7f7f57200640 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.156+0000 7f7f57200640 1 -- 172.21.15.103:0/658707059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 msgr2=0x5609e96f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.156+0000 7f7f57200640 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.156+0000 7f7f57200640 1 -- 172.21.15.103:0/658707059 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e93beb90 con 0x5609e96f8ba0 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.156+0000 7f7f57200640 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96f5ea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7f5001b8d0 tx=0x7f7f5001b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aae382dfd1d3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.156+0000 7f7f37e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:24:11.159 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.157+0000 7f7f37e00640 1 -- 172.21.15.103:0/658707059 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50003b70 con 0x5609e96f8ba0 2024-09-11T00:24:11.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.157+0000 7f7f37e00640 1 -- 172.21.15.103:0/658707059 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7f50003d10 con 0x5609e96f8ba0 2024-09-11T00:24:11.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.157+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f50011c90 con 0x5609e96f8ba0 2024-09-11T00:24:11.160 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.157+0000 7f7f37e00640 1 -- 172.21.15.103:0/658707059 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f50011440 con 0x5609e96f8ba0 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.157+0000 7f7f5feb4740 10 client.15417 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.157+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609e9707190 con 0x5609e96f8ba0 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f37e00640 1 -- 172.21.15.103:0/658707059 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f50011c90 con 0x5609e96f8ba0 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f37e00640 1 -- 172.21.15.103:0/658707059 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7f50004550 con 0x5609e96f8ba0 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f5feb4740 10 client.15417 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f5feb4740 10 client.15417 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f5feb4740 20 client.15417 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e97084e0 0x5609e970a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f5feb4740 4 client.15417 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.158+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609e9663bf0 con 0x5609e97084e0 2024-09-11T00:24:11.161 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.159+0000 7f7f55e00640 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e97084e0 0x5609e970a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.159+0000 7f7f55e00640 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e97084e0 0x5609e970a8d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75edc58bc2d37fae server_cookie=2578810ffee1285c in_seq=0 out_seq=0 2024-09-11T00:24:11.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.159+0000 7f7f37e00640 10 client.15417 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:11.162 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.160+0000 7f7f37e00640 1 -- 172.21.15.103:0/658707059 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5609e9663bf0 con 0x5609e97084e0 2024-09-11T00:24:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.160+0000 7f7f37e00640 10 client.15417 handle_command_reply: tid=0 2024-09-11T00:24:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 client.15417 shutdown 2024-09-11T00:24:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 msgr2=0x5609e96f5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96f5ea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7f5001b8d0 tx=0x7f7f5001b730 comp rx=0 tx=0).stop 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 shutdown_connections 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96f8ba0 0x5609e96f5ea0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e97084e0 0x5609e970a8d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/658707059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 >> 172.21.15.103:0/658707059 conn(0x5609e96ea120 msgr2=0x5609e96efc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.161+0000 7f7f37e00640 0 client.15417 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.162+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 shutdown_connections 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.162+0000 7f7f5feb4740 1 -- 172.21.15.103:0/658707059 wait complete. 2024-09-11T00:24:11.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.162+0000 7f7f5feb4740 20 client.15417 trim_cache size 0 max 16384 2024-09-11T00:24:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 Processor -- start 2024-09-11T00:24:11.182 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 -- start start 2024-09-11T00:24:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ea120 0x5609e9704ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581b8770 con 0x5609e96ea120 2024-09-11T00:24:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581b6040 con 0x5609e96ecc10 2024-09-11T00:24:11.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.180+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581b3e90 con 0x5609e96ec2f0 2024-09-11T00:24:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.181+0000 7f7f57200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ea120 0x5609e9704ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.181+0000 7f7f55e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.181+0000 7f7f55e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39262/0 (socket says 172.21.15.103:39262) 2024-09-11T00:24:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.181+0000 7f7f55e00640 1 -- 172.21.15.103:0/555340373 learned_addr learned my addr 172.21.15.103:0/555340373 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:11.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.182+0000 7f7f55e00640 1 -- 172.21.15.103:0/555340373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 msgr2=0x5609e96f9f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:24:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.182+0000 7f7f55e00640 1 --2- 172.21.15.103:0/555340373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f9f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.182+0000 7f7f55e00640 1 -- 172.21.15.103:0/555340373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ea120 msgr2=0x5609e9704ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.182+0000 7f7f55e00640 1 --2- 172.21.15.103:0/555340373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ea120 0x5609e9704ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.182+0000 7f7f55e00640 1 -- 172.21.15.103:0/555340373 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609e96eae00 con 0x5609e96ec2f0 2024-09-11T00:24:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.183+0000 7f7f55e00640 1 --2- 172.21.15.103:0/555340373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f7600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f440092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2704408474edc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.183+0000 7f7f42a00640 1 -- 172.21.15.103:0/555340373 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44023620 con 0x5609e96ec2f0 2024-09-11T00:24:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.183+0000 7f7f42a00640 1 -- 172.21.15.103:0/555340373 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4401e030 con 0x5609e96ec2f0 2024-09-11T00:24:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.183+0000 7f7f42a00640 1 -- 172.21.15.103:0/555340373 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44011730 con 0x5609e96ec2f0 2024-09-11T00:24:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 -- 172.21.15.103:0/555340373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 msgr2=0x5609e96f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/555340373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f7600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7f44005fd0 tx=0x7f7f440092f0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 -- 172.21.15.103:0/555340373 shutdown_connections 2024-09-11T00:24:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/555340373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ecc10 0x5609e96f9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/555340373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96f7600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/555340373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ea120 0x5609e9704ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 -- 172.21.15.103:0/555340373 >> 172.21.15.103:0/555340373 conn(0x5609e96f5490 msgr2=0x5609e96f6ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:11.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.184+0000 7f7f5feb4740 1 -- 172.21.15.103:0/555340373 shutdown_connections 2024-09-11T00:24:11.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.185+0000 7f7f5feb4740 1 -- 172.21.15.103:0/555340373 wait complete. 2024-09-11T00:24:11.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.185+0000 7f7f5feb4740 1 -- start start 2024-09-11T00:24:11.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.185+0000 7f7f5feb4740 1 Processor -- start 2024-09-11T00:24:11.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ea120 0x5609e96fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f5feb4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ecc10 0x5609e976f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581b4500 con 0x5609e96ecc10 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581b5970 con 0x5609e96ea120 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f5feb4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581b4bd0 con 0x5609e96ec2f0 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f5feb4740 10 client.0 _ll_register_callbacks cb 0x7f7f50009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f57200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ea120 0x5609e96fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f55e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f55e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96fc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39266/0 (socket says 172.21.15.103:39266) 2024-09-11T00:24:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.186+0000 7f7f55e00640 1 -- 172.21.15.103:0/288826033 learned_addr learned my addr 172.21.15.103:0/288826033 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.187+0000 7f7f57200640 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ecc10 0x5609e976f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.190 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.187+0000 7f7f55e00640 1 -- 172.21.15.103:0/288826033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ea120 msgr2=0x5609e96fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.187+0000 7f7f55e00640 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ea120 0x5609e96fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.187+0000 7f7f55e00640 1 -- 172.21.15.103:0/288826033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ecc10 msgr2=0x5609e976f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.187+0000 7f7f55e00640 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ecc10 0x5609e976f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.187+0000 7f7f55e00640 1 -- 172.21.15.103:0/288826033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609e976f990 con 0x5609e96ec2f0 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.188+0000 7f7f55e00640 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96fc960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f440057c0 tx=0x7f7f4401b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d870e170542345d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.188+0000 7f7f37e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.188+0000 7f7f37e00640 1 -- 172.21.15.103:0/288826033 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44005050 con 0x5609e96ec2f0 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.188+0000 7f7f37e00640 1 -- 172.21.15.103:0/288826033 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7f440051f0 con 0x5609e96ec2f0 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.188+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4401e030 con 0x5609e96ec2f0 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.189+0000 7f7f37e00640 1 -- 172.21.15.103:0/288826033 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7f44035050 con 0x5609e96ec2f0 2024-09-11T00:24:11.191 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.189+0000 7f7f5feb4740 10 client.25465 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:11.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.189+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5609e9776d10 con 0x5609e96ec2f0 2024-09-11T00:24:11.192 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.189+0000 7f7f37e00640 1 -- 172.21.15.103:0/288826033 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f4401bbb0 con 0x5609e96ec2f0 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.190+0000 7f7f37e00640 1 -- 172.21.15.103:0/288826033 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7f4400f040 con 0x5609e96ec2f0 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.190+0000 7f7f5feb4740 10 client.25465 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.190+0000 7f7f5feb4740 10 client.25465 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.190+0000 7f7f5feb4740 20 client.25465 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.190+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e96f8ba0 0x5609e9779460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.190+0000 7f7f5feb4740 4 client.25465 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.190+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x7f7f4401e030 con 0x5609e96f8ba0 2024-09-11T00:24:11.193 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.191+0000 7f7f57200640 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e96f8ba0 0x5609e9779460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:11.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.191+0000 7f7f37e00640 10 client.25465 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:11.194 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.191+0000 7f7f57200640 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e96f8ba0 0x5609e9779460 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fad47d233eac62 server_cookie=7953f9148abd421f in_seq=0 out_seq=0 2024-09-11T00:24:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f37e00640 1 -- 172.21.15.103:0/288826033 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f7f4401e030 con 0x5609e96f8ba0 2024-09-11T00:24:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f37e00640 10 client.25465 handle_command_reply: tid=0 2024-09-11T00:24:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 client.25465 shutdown 2024-09-11T00:24:11.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 msgr2=0x5609e96fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96fc960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f440057c0 tx=0x7f7f4401b590 comp rx=0 tx=0).stop 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 shutdown_connections 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5609e96ecc10 0x5609e976f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5609e96f8ba0 0x5609e9779460 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5609e96ec2f0 0x5609e96fc960 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7f440057c0 tx=0x7f7f4401b590 comp rx=0 tx=0).stop 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 --2- 172.21.15.103:0/288826033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5609e96ea120 0x5609e96fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.205+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 >> 172.21.15.103:0/288826033 conn(0x5609e96f5490 msgr2=0x5609e96f7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:11.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.206+0000 7f7f37e00640 0 client.25465 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.206+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 shutdown_connections 2024-09-11T00:24:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.206+0000 7f7f5feb4740 1 -- 172.21.15.103:0/288826033 wait complete. 2024-09-11T00:24:11.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.207+0000 7f7f5feb4740 20 client.25465 trim_cache size 0 max 16384 2024-09-11T00:24:11.209 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 2, 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "21e61e96": { 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 2, 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:24:11.210 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.2, 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:11.211 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:11.212 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:11.212 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:24:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7f3c0b5490 msgr2=0x7f7f3c0b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7f3c0b5490 0x7f7f3c0b7950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7f48006010 tx=0x7f7f480c9290 comp rx=0 tx=0).stop 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f58103b10 msgr2=0x7f7f5810c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f58103b10 0x7f7f5810c590 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f48005940 tx=0x7f7f48004270 comp rx=0 tx=0).stop 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 shutdown_connections 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7f3c0b5490 0x7f7f3c0b7950 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f5810cf70 0x7f7f58109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7f58104430 0x7f7f581097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 --2- 172.21.15.103:0/3375741818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7f58103b10 0x7f7f5810c590 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 >> 172.21.15.103:0/3375741818 conn(0x7f7f580fd5b0 msgr2=0x7f7f5810f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.208+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 shutdown_connections 2024-09-11T00:24:11.213 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:11.209+0000 7f7f5ca00640 1 -- 172.21.15.103:0/3375741818 wait complete. 2024-09-11T00:24:12.251 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '21e61e96', quiesce took 4.9 seconds. Will release after: 20.299999999999997 2024-09-11T00:24:12.484 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:12 smithi103 ceph-mon[30578]: pgmap v432: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T00:24:12.739 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:12 smithi117 ceph-mon[35525]: pgmap v432: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T00:24:12.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:12 smithi183 ceph-mon[35934]: pgmap v432: 129 pgs: 129 active+clean; 661 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-11T00:24:14.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:14 smithi183 ceph-mon[35934]: pgmap v433: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:24:14.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:14 smithi103 ceph-mon[30578]: pgmap v433: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:24:14.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:14 smithi117 ceph-mon[35525]: pgmap v433: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:24:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:16 smithi183 ceph-mon[35934]: pgmap v434: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T00:24:16.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:16 smithi103 ceph-mon[30578]: pgmap v434: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T00:24:16.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:16 smithi117 ceph-mon[35525]: pgmap v434: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 938 B/s wr, 6 op/s 2024-09-11T00:24:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:18 smithi183 ceph-mon[35934]: pgmap v435: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-11T00:24:18.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:18 smithi103 ceph-mon[30578]: pgmap v435: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-11T00:24:18.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:18 smithi117 ceph-mon[35525]: pgmap v435: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-11T00:24:20.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:20 smithi183 ceph-mon[35934]: pgmap v436: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:24:20.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:20 smithi103 ceph-mon[30578]: pgmap v436: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:24:20.940 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:20 smithi117 ceph-mon[35525]: pgmap v436: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 938 B/s wr, 7 op/s 2024-09-11T00:24:22.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 -- 172.21.15.103:0/899613873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee81049d0 msgr2=0x7f5ee8104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 --2- 172.21.15.103:0/899613873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee81049d0 0x7f5ee8104dd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009930 tx=0x7f5ed802a420 comp rx=0 tx=0).stop 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 -- 172.21.15.103:0/899613873 shutdown_connections 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 --2- 172.21.15.103:0/899613873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee81064e0 0x7f5ee810ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 --2- 172.21.15.103:0/899613873 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ee8105b20 0x7f5ee8105fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 --2- 172.21.15.103:0/899613873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee81049d0 0x7f5ee8104dd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 -- 172.21.15.103:0/899613873 >> 172.21.15.103:0/899613873 conn(0x7f5ee8100270 msgr2=0x7f5ee81026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 -- 172.21.15.103:0/899613873 shutdown_connections 2024-09-11T00:24:22.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.323+0000 7f5ee7200640 1 -- 172.21.15.103:0/899613873 wait complete. 2024-09-11T00:24:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.324+0000 7f5ee7200640 1 Processor -- start 2024-09-11T00:24:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.324+0000 7f5ee7200640 1 -- start start 2024-09-11T00:24:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ee81049d0 0x7f5ee819add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee8105b20 0x7f5ee819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee7200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee81064e0 0x7f5ee819c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8110620 con 0x7f5ee81049d0 2024-09-11T00:24:22.328 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee7200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81104a0 con 0x7f5ee81064e0 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee7200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81107a0 con 0x7f5ee8105b20 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ee81049d0 0x7f5ee819add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee8105b20 0x7f5ee819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee8105b20 0x7f5ee819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:33050/0 (socket says 172.21.15.103:33050) 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee5400640 1 -- 172.21.15.103:0/3443879554 learned_addr learned my addr 172.21.15.103:0/3443879554 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.325+0000 7f5ee6800640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee81064e0 0x7f5ee819c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee5400640 1 -- 172.21.15.103:0/3443879554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee81064e0 msgr2=0x7f5ee819c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee5400640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee81064e0 0x7f5ee819c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee5400640 1 -- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ee81049d0 msgr2=0x7f5ee819add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee5400640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ee81049d0 0x7f5ee819add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee5400640 1 -- 172.21.15.103:0/3443879554 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee819c910 con 0x7f5ee8105b20 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee5400640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee8105b20 0x7f5ee819b420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e9f0 tx=0x7f5ed400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:22.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5edea00640 1 -- 172.21.15.103:0/3443879554 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400cd80 con 0x7f5ee8105b20 2024-09-11T00:24:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee819cc00 con 0x7f5ee8105b20 2024-09-11T00:24:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5edea00640 1 -- 172.21.15.103:0/3443879554 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed40108e0 con 0x7f5ee8105b20 2024-09-11T00:24:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5edea00640 1 -- 172.21.15.103:0/3443879554 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4018980 con 0x7f5ee8105b20 2024-09-11T00:24:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.326+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee81a32d0 con 0x7f5ee8105b20 2024-09-11T00:24:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.330+0000 7f5eefcd7740 1 Processor -- start 2024-09-11T00:24:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5eefcd7740 1 -- start start 2024-09-11T00:24:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf031d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf0346a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558bbef916b0 con 0x558bbf034cf0 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558bbef80ae0 con 0x558bbf0322b0 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bbef848f0 con 0x558bbf02f980 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf031d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf031d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:33056/0 (socket says 172.21.15.103:33056) 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5ee6800640 1 -- 172.21.15.103:0/956389527 learned_addr learned my addr 172.21.15.103:0/956389527 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.331+0000 7f5edea00640 1 -- 172.21.15.103:0/3443879554 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5ed4010430 con 0x7f5ee8105b20 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf0346a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee6800640 1 -- 172.21.15.103:0/956389527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 msgr2=0x558bbf0346a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee6800640 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf0346a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee6800640 1 -- 172.21.15.103:0/956389527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 msgr2=0x558bbf0371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee5400640 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee6800640 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee6800640 1 -- 172.21.15.103:0/956389527 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbef4d780 con 0x558bbf02f980 2024-09-11T00:24:22.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.332+0000 7f5ee5400640 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:24:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.333+0000 7f5ee6800640 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf031d70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009f90 tx=0x7f5ee00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=956b8727b740ee60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.333+0000 7f5ed3400640 1 -- 172.21.15.103:0/956389527 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004d20 con 0x558bbf02f980 2024-09-11T00:24:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.333+0000 7f5ed3400640 1 -- 172.21.15.103:0/956389527 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0004050 con 0x558bbf02f980 2024-09-11T00:24:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.333+0000 7f5ed3400640 1 -- 172.21.15.103:0/956389527 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0005280 con 0x558bbf02f980 2024-09-11T00:24:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.333+0000 7f5edea00640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5ecc0b52e0 0x7f5ecc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5edea00640 1 -- 172.21.15.103:0/3443879554 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ed40e62b0 con 0x7f5ee8105b20 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 -- 172.21.15.103:0/956389527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 msgr2=0x558bbf031d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf031d70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009f90 tx=0x7f5ee00118d0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5ecc0b52e0 0x7f5ecc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 -- 172.21.15.103:0/956389527 shutdown_connections 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf0346a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/956389527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf031d70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 -- 172.21.15.103:0/956389527 >> 172.21.15.103:0/956389527 conn(0x558bbf019ac0 msgr2=0x558bbf01a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 -- 172.21.15.103:0/956389527 shutdown_connections 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.334+0000 7f5eefcd7740 1 -- 172.21.15.103:0/956389527 wait complete. 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.335+0000 7f5eefcd7740 1 -- start start 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.335+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5ecc0b52e0 0x7f5ecc0b77a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8006010 tx=0x7f5ed80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:22.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.335+0000 7f5eefcd7740 1 Processor -- start 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf0391d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf039870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf03a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558bbeeb8ab0 con 0x558bbf034cf0 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558bbef27570 con 0x558bbf0322b0 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558bbf037cd0 con 0x558bbf02f980 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5eefcd7740 10 client.0 _ll_register_callbacks cb 0x558bbef36100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf039870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf03a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf039870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48538/0 (socket says 172.21.15.103:48538) 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 -- 172.21.15.103:0/389939630 learned_addr learned my addr 172.21.15.103:0/389939630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf0391d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 -- 172.21.15.103:0/389939630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 msgr2=0x558bbf0391d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf0391d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 -- 172.21.15.103:0/389939630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 msgr2=0x558bbf03a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf03a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.336+0000 7f5ee5400640 1 -- 172.21.15.103:0/389939630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbece2790 con 0x558bbf0322b0 2024-09-11T00:24:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5ee5400640 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf039870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40e6550 tx=0x7f5ed40ac080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37afc8d60c964c47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5ec7e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:24:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5ec7e00640 1 -- 172.21.15.103:0/389939630 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40abb90 con 0x558bbf0322b0 2024-09-11T00:24:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee0004050 con 0x558bbf0322b0 2024-09-11T00:24:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5ec7e00640 1 -- 172.21.15.103:0/389939630 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ed4018b20 con 0x558bbf0322b0 2024-09-11T00:24:22.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5ec7e00640 1 -- 172.21.15.103:0/389939630 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ed40ac730 con 0x558bbf0322b0 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5eefcd7740 10 client.25445 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.337+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bbed5b620 con 0x558bbf0322b0 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5ec7e00640 1 -- 172.21.15.103:0/389939630 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0004050 con 0x558bbf0322b0 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5ec7e00640 1 -- 172.21.15.103:0/389939630 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5ed40a8080 con 0x558bbf0322b0 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5eefcd7740 10 client.25445 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5eefcd7740 10 client.25445 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5eefcd7740 20 client.25445 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf044920 0x558bbf046d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5eefcd7740 4 client.25445 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.338+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bbef93600 con 0x558bbf044920 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.339+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf044920 0x558bbf046d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.339+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf044920 0x558bbf046d30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b19d7edeef98f06 server_cookie=adbeab13caab6f31 in_seq=0 out_seq=0 2024-09-11T00:24:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.339+0000 7f5ec7e00640 10 client.25445 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5ec7e00640 1 -- 172.21.15.103:0/389939630 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558bbef93600 con 0x558bbf044920 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5ec7e00640 10 client.25445 handle_command_reply: tid=0 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 client.25445 shutdown 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 msgr2=0x558bbf039870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf039870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40e6550 tx=0x7f5ed40ac080 comp rx=0 tx=0).stop 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 shutdown_connections 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf03a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf0322b0 0x558bbf039870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf044920 0x558bbf046d30 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/389939630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf02f980 0x558bbf0391d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 >> 172.21.15.103:0/389939630 conn(0x558bbf019ac0 msgr2=0x558bbf018bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.340+0000 7f5ec7e00640 0 client.25445 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.341+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 shutdown_connections 2024-09-11T00:24:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.341+0000 7f5eefcd7740 1 -- 172.21.15.103:0/389939630 wait complete. 2024-09-11T00:24:22.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.341+0000 7f5eefcd7740 20 client.25445 trim_cache size 0 max 16384 2024-09-11T00:24:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 Processor -- start 2024-09-11T00:24:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 -- start start 2024-09-11T00:24:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf0357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf0322b0 0x558bbf0380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81a7740 con 0x558bbf034cf0 2024-09-11T00:24:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81a5010 con 0x558bbf02f980 2024-09-11T00:24:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.356+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee8074d40 con 0x558bbf0322b0 2024-09-11T00:24:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf0357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0385e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35910/0 (socket says 172.21.15.103:35910) 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee6800640 1 -- 172.21.15.103:0/2591387470 learned_addr learned my addr 172.21.15.103:0/2591387470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5400640 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf0322b0 0x558bbf0380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5e00640 1 -- 172.21.15.103:0/2591387470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf0322b0 msgr2=0x558bbf0380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf0322b0 0x558bbf0380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5e00640 1 -- 172.21.15.103:0/2591387470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 msgr2=0x558bbf0385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5e00640 1 -- 172.21.15.103:0/2591387470 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bbf0320e0 con 0x558bbf02f980 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee6800640 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.357+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf0357e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8005ab0 tx=0x7f5ed8040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b18347657fef49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.358+0000 7f5ed2a00640 1 -- 172.21.15.103:0/2591387470 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8035040 con 0x558bbf02f980 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.358+0000 7f5ed2a00640 1 -- 172.21.15.103:0/2591387470 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed8003b60 con 0x558bbf02f980 2024-09-11T00:24:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.358+0000 7f5ed2a00640 1 -- 172.21.15.103:0/2591387470 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8051a80 con 0x558bbf02f980 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.358+0000 7f5eefcd7740 1 -- 172.21.15.103:0/2591387470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 msgr2=0x558bbf0357e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.358+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf0357e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8005ab0 tx=0x7f5ed8040000 comp rx=0 tx=0).stop 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 -- 172.21.15.103:0/2591387470 shutdown_connections 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf034cf0 0x558bbf0385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf0322b0 0x558bbf0380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/2591387470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf0357e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 -- 172.21.15.103:0/2591387470 >> 172.21.15.103:0/2591387470 conn(0x558bbf019ac0 msgr2=0x558bbf0350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 -- 172.21.15.103:0/2591387470 shutdown_connections 2024-09-11T00:24:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 -- 172.21.15.103:0/2591387470 wait complete. 2024-09-11T00:24:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 -- start start 2024-09-11T00:24:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.359+0000 7f5eefcd7740 1 Processor -- start 2024-09-11T00:24:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf040a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf0322b0 0x558bbf03f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5eefcd7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf034cf0 0x558bbf03fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81a34f0 con 0x558bbf0322b0 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81a4940 con 0x558bbf02f980 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5eefcd7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee81a3ba0 con 0x558bbf034cf0 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5eefcd7740 10 client.0 _ll_register_callbacks cb 0x7f5ed4009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.360+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf0322b0 0x558bbf03f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf0322b0 0x558bbf03f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:35912/0 (socket says 172.21.15.103:35912) 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee6800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf034cf0 0x558bbf03fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5400640 1 -- 172.21.15.103:0/4106600232 learned_addr learned my addr 172.21.15.103:0/4106600232 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5e00640 1 -- 172.21.15.103:0/4106600232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf034cf0 msgr2=0x558bbf03fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf034cf0 0x558bbf03fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5e00640 1 -- 172.21.15.103:0/4106600232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf0322b0 msgr2=0x558bbf03f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf0322b0 0x558bbf03f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5e00640 1 -- 172.21.15.103:0/4106600232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bbf0402e0 con 0x558bbf02f980 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.361+0000 7f5ee5400640 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf0322b0 0x558bbf03f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5ee5e00640 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf040a30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8005c40 tx=0x7f5ed8009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d5734375b2043 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5ec7e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5ec7e00640 1 -- 172.21.15.103:0/4106600232 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8035070 con 0x558bbf02f980 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ed8003b60 con 0x558bbf02f980 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5ec7e00640 1 -- 172.21.15.103:0/4106600232 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5ed8003d10 con 0x558bbf02f980 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5ec7e00640 1 -- 172.21.15.103:0/4106600232 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ed80036b0 con 0x558bbf02f980 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5eefcd7740 10 client.25451 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.362+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558bbf0b1840 con 0x558bbf02f980 2024-09-11T00:24:22.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5ec7e00640 1 -- 172.21.15.103:0/4106600232 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ed8003b60 con 0x558bbf02f980 2024-09-11T00:24:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5ec7e00640 1 -- 172.21.15.103:0/4106600232 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5ed8058070 con 0x558bbf02f980 2024-09-11T00:24:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5eefcd7740 10 client.25451 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5eefcd7740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5eefcd7740 20 client.25451 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:22.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf0b2050 0x558bbf0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5eefcd7740 4 client.25451 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558bbef8f0b0 con 0x558bbf0b2050 2024-09-11T00:24:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.363+0000 7f5ee5400640 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf0b2050 0x558bbf0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.364+0000 7f5ee5400640 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf0b2050 0x558bbf0b4470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7407204d316a70 server_cookie=b30378dd05dccbb2 in_seq=0 out_seq=0 2024-09-11T00:24:22.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.364+0000 7f5ec7e00640 10 client.25451 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5ec7e00640 1 -- 172.21.15.103:0/4106600232 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558bbef8f0b0 con 0x558bbf0b2050 2024-09-11T00:24:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5ec7e00640 10 client.25451 handle_command_reply: tid=0 2024-09-11T00:24:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 client.25451 shutdown 2024-09-11T00:24:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 msgr2=0x558bbf040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf040a30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8005c40 tx=0x7f5ed8009290 comp rx=0 tx=0).stop 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 shutdown_connections 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558bbf0b2050 0x558bbf0b4470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558bbf034cf0 0x558bbf03fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bbf0322b0 0x558bbf03f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5ec7e00640 0 client.25451 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 --2- 172.21.15.103:0/4106600232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558bbf02f980 0x558bbf040a30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 >> 172.21.15.103:0/4106600232 conn(0x558bbf019ac0 msgr2=0x558bbf01db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 shutdown_connections 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 1 -- 172.21.15.103:0/4106600232 wait complete. 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.372+0000 7f5eefcd7740 20 client.25451 trim_cache size 0 max 16384 2024-09-11T00:24:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:24:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:24:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:24:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:24:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5ecc0b52e0 msgr2=0x7f5ecc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5ecc0b52e0 0x7f5ecc0b77a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8006010 tx=0x7f5ed80314a0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee8105b20 msgr2=0x7f5ee819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee8105b20 0x7f5ee819b420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400e9f0 tx=0x7f5ed400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 shutdown_connections 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5ecc0b52e0 0x7f5ecc0b77a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ee81064e0 0x7f5ee819c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ee8105b20 0x7f5ee819b420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 --2- 172.21.15.103:0/3443879554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5ee81049d0 0x7f5ee819add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.373+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 >> 172.21.15.103:0/3443879554 conn(0x7f5ee8100270 msgr2=0x7f5ee8108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.374+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 shutdown_connections 2024-09-11T00:24:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:22.374+0000 7f5ee7200640 1 -- 172.21.15.103:0/3443879554 wait complete. 2024-09-11T00:24:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:22 smithi183 ceph-mon[35934]: pgmap v437: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-11T00:24:22.856 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:22 smithi103 ceph-mon[30578]: pgmap v437: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-11T00:24:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:22 smithi117 ceph-mon[35525]: pgmap v437: 129 pgs: 129 active+clean; 667 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-11T00:24:24.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:24 smithi183 ceph-mon[35934]: pgmap v438: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-11T00:24:24.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:24 smithi103 ceph-mon[30578]: pgmap v438: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-11T00:24:24.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:24 smithi117 ceph-mon[35525]: pgmap v438: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-09-11T00:24:26.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:26 smithi183 ceph-mon[35934]: pgmap v439: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T00:24:26.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:26 smithi103 ceph-mon[30578]: pgmap v439: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T00:24:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:26 smithi117 ceph-mon[35525]: pgmap v439: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-11T00:24:28.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:28 smithi183 ceph-mon[35934]: pgmap v440: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-11T00:24:28.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:28 smithi103 ceph-mon[30578]: pgmap v440: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-11T00:24:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:28 smithi117 ceph-mon[35525]: pgmap v440: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-11T00:24:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:30 smithi183 ceph-mon[35934]: pgmap v441: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-11T00:24:30.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:30 smithi103 ceph-mon[30578]: pgmap v441: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-11T00:24:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:30 smithi117 ceph-mon[35525]: pgmap v441: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-11T00:24:32.553 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:24:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:32 smithi183 ceph-mon[35934]: pgmap v442: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T00:24:32.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:32 smithi103 ceph-mon[30578]: pgmap v442: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T00:24:32.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:32 smithi117 ceph-mon[35525]: pgmap v442: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-11T00:24:33.014 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.796+0000 7f95ae000640 1 -- 172.21.15.103:0/2513973626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 msgr2=0x7f95a810ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.796+0000 7f95ae000640 1 --2- 172.21.15.103:0/2513973626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a810ac60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9594009900 tx=0x7f959402a410 comp rx=0 tx=0).stop 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.797+0000 7f95ae000640 1 -- 172.21.15.103:0/2513973626 shutdown_connections 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.797+0000 7f95ae000640 1 --2- 172.21.15.103:0/2513973626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95a810b1a0 0x7f95a81119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.797+0000 7f95ae000640 1 --2- 172.21.15.103:0/2513973626 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a810ac60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.797+0000 7f95ae000640 1 --2- 172.21.15.103:0/2513973626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a8104e10 0x7f95a81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.797+0000 7f95ae000640 1 -- 172.21.15.103:0/2513973626 >> 172.21.15.103:0/2513973626 conn(0x7f95a8100840 msgr2=0x7f95a8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:33.800 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.797+0000 7f95ae000640 1 -- 172.21.15.103:0/2513973626 shutdown_connections 2024-09-11T00:24:33.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.797+0000 7f95ae000640 1 -- 172.21.15.103:0/2513973626 wait complete. 2024-09-11T00:24:33.801 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 Processor -- start 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 -- start start 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95a8104e10 0x7f95a80776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a8077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a810b1a0 0x7f95a8078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8114e20 con 0x7f95a8104e10 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8114ca0 con 0x7f95a810b1a0 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.799+0000 7f95ae000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8114fa0 con 0x7f95a810a820 2024-09-11T00:24:33.802 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.800+0000 7f95a7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a8077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.800+0000 7f95a7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a8077bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:37876/0 (socket says 172.21.15.103:37876) 2024-09-11T00:24:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.800+0000 7f95a7e00640 1 -- 172.21.15.103:0/2855006780 learned_addr learned my addr 172.21.15.103:0/2855006780 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.800+0000 7f95acc00640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95a8104e10 0x7f95a80776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.800+0000 7f95ad600640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a810b1a0 0x7f95a8078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:33.803 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.801+0000 7f95a7e00640 1 -- 172.21.15.103:0/2855006780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a810b1a0 msgr2=0x7f95a8078130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:33.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.801+0000 7f95a7e00640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a810b1a0 0x7f95a8078130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:33.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.801+0000 7f95a7e00640 1 -- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95a8104e10 msgr2=0x7f95a80776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:33.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.801+0000 7f95a7e00640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95a8104e10 0x7f95a80776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:33.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.801+0000 7f95a7e00640 1 -- 172.21.15.103:0/2855006780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a81ab820 con 0x7f95a810a820 2024-09-11T00:24:33.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.801+0000 7f95acc00640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95a8104e10 0x7f95a80776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:33.804 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.802+0000 7f95a7e00640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a8077bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9594005940 tx=0x7f95940040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:33.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.802+0000 7f95a5600640 1 -- 172.21.15.103:0/2855006780 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959403f030 con 0x7f95a810a820 2024-09-11T00:24:33.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.802+0000 7f95a5600640 1 -- 172.21.15.103:0/2855006780 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940044c0 con 0x7f95a810a820 2024-09-11T00:24:33.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.802+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a81abab0 con 0x7f95a810a820 2024-09-11T00:24:33.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.802+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a81abfe0 con 0x7f95a810a820 2024-09-11T00:24:33.805 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.802+0000 7f95a5600640 1 -- 172.21.15.103:0/2855006780 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594035070 con 0x7f95a810a820 2024-09-11T00:24:33.806 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.804+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95880052c0 con 0x7f95a810a820 2024-09-11T00:24:33.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.808+0000 7f95a5600640 1 -- 172.21.15.103:0/2855006780 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f959404c060 con 0x7f95a810a820 2024-09-11T00:24:33.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.810+0000 7f95a5600640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f95900b5410 0x7f95900b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:33.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.811+0000 7f95a5600640 1 -- 172.21.15.103:0/2855006780 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9594105ee0 con 0x7f95a810a820 2024-09-11T00:24:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.811+0000 7f95a5600640 1 -- 172.21.15.103:0/2855006780 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f95941063a0 con 0x7f95a810a820 2024-09-11T00:24:33.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.811+0000 7f95acc00640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f95900b5410 0x7f95900b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:33.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:33.812+0000 7f95acc00640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f95900b5410 0x7f95900b78d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f959c009fc0 tx=0x7f959c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:34.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.202+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95880055b0 con 0x7f95a810a820 2024-09-11T00:24:34.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.203+0000 7f95a5600640 1 -- 172.21.15.103:0/2855006780 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11933 (secure 0 0 0) 0x7f95940d51a0 con 0x7f95a810a820 2024-09-11T00:24:34.205 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:24:34.205 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":27,"btime":"2024-09-11T00:23:29:477265+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:23:28.708651+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24622":{"gid":24622,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3946403859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3946403859},{"type":"v1","addr":"172.21.15.183:6835","nonce":3946403859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:24:34.206 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 27 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f95900b5410 msgr2=0x7f95900b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f95900b5410 0x7f95900b78d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f959c009fc0 tx=0x7f959c005f90 comp rx=0 tx=0).stop 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 msgr2=0x7f95a8077bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a8077bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9594005940 tx=0x7f95940040c0 comp rx=0 tx=0).stop 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 shutdown_connections 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f95900b5410 0x7f95900b78d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f95a810b1a0 0x7f95a8078130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a810a820 0x7f95a8077bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 --2- 172.21.15.103:0/2855006780 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95a8104e10 0x7f95a80776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:34.209 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 >> 172.21.15.103:0/2855006780 conn(0x7f95a8100840 msgr2=0x7f95a8102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:34.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 shutdown_connections 2024-09-11T00:24:34.210 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:34.207+0000 7f95ae000640 1 -- 172.21.15.103:0/2855006780 wait complete. 2024-09-11T00:24:34.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:34 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2855006780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:34.592 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:34 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2855006780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:34 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2855006780' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:24:35.143 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --set-id 21e61e96 --release --await' 2024-09-11T00:24:35.143 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --set-id 21e61e96 --release --await 2024-09-11T00:24:35.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:35 smithi103 ceph-mon[30578]: pgmap v443: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:24:35.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:35 smithi183 ceph-mon[35934]: pgmap v443: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:24:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 -- 172.21.15.103:0/1383098373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694103ad0 msgr2=0x7fe694103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 --2- 172.21.15.103:0/1383098373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694103ad0 0x7fe694103f50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe680009930 tx=0x7fe68002a440 comp rx=0 tx=0).stop 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 -- 172.21.15.103:0/1383098373 shutdown_connections 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 --2- 172.21.15.103:0/1383098373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694104490 0x7fe69410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 --2- 172.21.15.103:0/1383098373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694103ad0 0x7fe694103f50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 --2- 172.21.15.103:0/1383098373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102980 0x7fe694102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 -- 172.21.15.103:0/1383098373 >> 172.21.15.103:0/1383098373 conn(0x7fe6940fe240 msgr2=0x7fe694100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.538+0000 7fe699800640 1 -- 172.21.15.103:0/1383098373 shutdown_connections 2024-09-11T00:24:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.539+0000 7fe699800640 1 -- 172.21.15.103:0/1383098373 wait complete. 2024-09-11T00:24:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.540+0000 7fe699800640 1 Processor -- start 2024-09-11T00:24:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.540+0000 7fe699800640 1 -- start start 2024-09-11T00:24:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.540+0000 7fe699800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102980 0x7fe694076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe699800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103ad0 0x7fe694075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe699800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694104490 0x7fe694075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe699800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410e620 con 0x7fe694103ad0 2024-09-11T00:24:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe699800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410e4a0 con 0x7fe694102980 2024-09-11T00:24:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe699800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe69410e7a0 con 0x7fe694104490 2024-09-11T00:24:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe692000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103ad0 0x7fe694075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694104490 0x7fe694075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694104490 0x7fe694075a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:37896/0 (socket says 172.21.15.103:37896) 2024-09-11T00:24:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 -- 172.21.15.103:0/433873366 learned_addr learned my addr 172.21.15.103:0/433873366 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe692a00640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102980 0x7fe694076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 -- 172.21.15.103:0/433873366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102980 msgr2=0x7fe694076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102980 0x7fe694076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 -- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103ad0 msgr2=0x7fe694075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103ad0 0x7fe694075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.541+0000 7fe693400640 1 -- 172.21.15.103:0/433873366 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6940762f0 con 0x7fe694104490 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.542+0000 7fe692a00640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102980 0x7fe694076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.542+0000 7fe693400640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694104490 0x7fe694075a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00eb10 tx=0x7fe68c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.542+0000 7fe68b400640 1 -- 172.21.15.103:0/433873366 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004b20 con 0x7fe694104490 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.542+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941a2e80 con 0x7fe694104490 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.542+0000 7fe68b400640 1 -- 172.21.15.103:0/433873366 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68c004cc0 con 0x7fe694104490 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.543+0000 7fe68b400640 1 -- 172.21.15.103:0/433873366 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c004b20 con 0x7fe694104490 2024-09-11T00:24:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.543+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a3340 con 0x7fe694104490 2024-09-11T00:24:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.546+0000 7fe68b400640 1 -- 172.21.15.103:0/433873366 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe68c005020 con 0x7fe694104490 2024-09-11T00:24:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.546+0000 7fe69c88c740 1 Processor -- start 2024-09-11T00:24:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe69c88c740 1 -- start start 2024-09-11T00:24:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3aab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3ab0c0 0x55a86d3ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3adb00 0x55a86d3affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe69c88c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a86d31b6b0 con 0x55a86d3ab0c0 2024-09-11T00:24:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe69c88c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a86d30aae0 con 0x55a86d3adb00 2024-09-11T00:24:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe69c88c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a86d30e8f0 con 0x55a86d3a8790 2024-09-11T00:24:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe692000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe692a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3ab0c0 0x55a86d3ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe692000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3aab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:37902/0 (socket says 172.21.15.103:37902) 2024-09-11T00:24:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.547+0000 7fe692000640 1 -- 172.21.15.103:0/470944247 learned_addr learned my addr 172.21.15.103:0/470944247 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe692000640 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3adb00 0x55a86d3affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe68b400640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6780b5560 0x7fe6780b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe68b400640 1 -- 172.21.15.103:0/433873366 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe68c0e61e0 con 0x7fe694104490 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe692a00640 1 -- 172.21.15.103:0/470944247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 msgr2=0x55a86d3aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe692a00640 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3aab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe692a00640 1 -- 172.21.15.103:0/470944247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3adb00 msgr2=0x55a86d3affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe692a00640 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3adb00 0x55a86d3affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.548+0000 7fe692a00640 1 -- 172.21.15.103:0/470944247 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86d2d7780 con 0x55a86d3ab0c0 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.549+0000 7fe693400640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6780b5560 0x7fe6780b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.549+0000 7fe692a00640 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3ab0c0 0x55a86d3ad4b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe684009c30 tx=0x7fe6840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c10f708ab531d2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.549+0000 7fe693400640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6780b5560 0x7fe6780b7a20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005ba0 tx=0x7fe68c009a70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.549+0000 7fe67fe00640 1 -- 172.21.15.103:0/470944247 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684012450 con 0x55a86d3ab0c0 2024-09-11T00:24:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.550+0000 7fe67fe00640 1 -- 172.21.15.103:0/470944247 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684012aa0 con 0x55a86d3ab0c0 2024-09-11T00:24:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.550+0000 7fe67fe00640 1 -- 172.21.15.103:0/470944247 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400fa80 con 0x55a86d3ab0c0 2024-09-11T00:24:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.550+0000 7fe69c88c740 1 -- 172.21.15.103:0/470944247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3ab0c0 msgr2=0x55a86d3ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.550+0000 7fe69c88c740 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3ab0c0 0x55a86d3ad4b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe684009c30 tx=0x7fe6840117b0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 -- 172.21.15.103:0/470944247 shutdown_connections 2024-09-11T00:24:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3adb00 0x55a86d3affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3ab0c0 0x55a86d3ad4b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 --2- 172.21.15.103:0/470944247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 -- 172.21.15.103:0/470944247 >> 172.21.15.103:0/470944247 conn(0x55a86d3a38e0 msgr2=0x55a86d3a4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 -- 172.21.15.103:0/470944247 shutdown_connections 2024-09-11T00:24:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 -- 172.21.15.103:0/470944247 wait complete. 2024-09-11T00:24:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.551+0000 7fe69c88c740 1 -- start start 2024-09-11T00:24:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 1 Processor -- start 2024-09-11T00:24:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3adb00 0x55a86d3b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3b3440 0x55a86d3b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a86d242ab0 con 0x55a86d3b3440 2024-09-11T00:24:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a86d2b1570 con 0x55a86d3a8790 2024-09-11T00:24:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a86d3b0ae0 con 0x55a86d3adb00 2024-09-11T00:24:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.552+0000 7fe69c88c740 10 client.0 _ll_register_callbacks cb 0x55a86d2c0100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.553+0000 7fe692a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3adb00 0x55a86d3b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.553+0000 7fe692000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.553+0000 7fe692a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3adb00 0x55a86d3b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:37904/0 (socket says 172.21.15.103:37904) 2024-09-11T00:24:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.553+0000 7fe692a00640 1 -- 172.21.15.103:0/2311266777 learned_addr learned my addr 172.21.15.103:0/2311266777 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.553+0000 7fe692000640 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3b3440 0x55a86d3b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.554+0000 7fe692a00640 1 -- 172.21.15.103:0/2311266777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3a8790 msgr2=0x55a86d3b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.554+0000 7fe692a00640 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.554+0000 7fe692a00640 1 -- 172.21.15.103:0/2311266777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3b3440 msgr2=0x55a86d3b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.554+0000 7fe692a00640 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3b3440 0x55a86d3b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.554+0000 7fe692a00640 1 -- 172.21.15.103:0/2311266777 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86d06c790 con 0x55a86d3adb00 2024-09-11T00:24:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.554+0000 7fe692000640 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3b3440 0x55a86d3b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.555+0000 7fe692a00640 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3adb00 0x55a86d3b2890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe684005830 tx=0x7fe684012e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18946f9d900636e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.555+0000 7fe67cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:24:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.555+0000 7fe67cc00640 1 -- 172.21.15.103:0/2311266777 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684012660 con 0x55a86d3adb00 2024-09-11T00:24:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.555+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684012aa0 con 0x55a86d3adb00 2024-09-11T00:24:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.555+0000 7fe67cc00640 1 -- 172.21.15.103:0/2311266777 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe684012800 con 0x55a86d3adb00 2024-09-11T00:24:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.556+0000 7fe67cc00640 1 -- 172.21.15.103:0/2311266777 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe684005040 con 0x55a86d3adb00 2024-09-11T00:24:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.556+0000 7fe69c88c740 10 client.25501 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.556+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a86d0e5620 con 0x55a86d3adb00 2024-09-11T00:24:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe67cc00640 1 -- 172.21.15.103:0/2311266777 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684012aa0 con 0x55a86d3adb00 2024-09-11T00:24:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe67cc00640 1 -- 172.21.15.103:0/2311266777 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe6840046c0 con 0x55a86d3adb00 2024-09-11T00:24:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe69c88c740 10 client.25501 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe69c88c740 10 client.25501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe69c88c740 20 client.25501 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d3bdc80 0x55a86d3c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe69c88c740 4 client.25501 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.557+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86d31d600 con 0x55a86d3bdc80 2024-09-11T00:24:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.558+0000 7fe692000640 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d3bdc80 0x55a86d3c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.558+0000 7fe67cc00640 10 client.25501 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.558+0000 7fe692000640 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d3bdc80 0x55a86d3c0070 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c63e271b1f5b1e server_cookie=23431e1489cbd6e5 in_seq=0 out_seq=0 2024-09-11T00:24:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.559+0000 7fe67cc00640 1 -- 172.21.15.103:0/2311266777 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a86d31d600 con 0x55a86d3bdc80 2024-09-11T00:24:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.559+0000 7fe67cc00640 10 client.25501 handle_command_reply: tid=0 2024-09-11T00:24:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.559+0000 7fe69c88c740 1 client.25501 shutdown 2024-09-11T00:24:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3adb00 msgr2=0x55a86d3b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3adb00 0x55a86d3b2890 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe684005830 tx=0x7fe684012e80 comp rx=0 tx=0).stop 2024-09-11T00:24:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 shutdown_connections 2024-09-11T00:24:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3b3440 0x55a86d3b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d3bdc80 0x55a86d3c0070 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3adb00 0x55a86d3b2890 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2311266777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 >> 172.21.15.103:0/2311266777 conn(0x55a86d3a38e0 msgr2=0x55a86d3a2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe67cc00640 0 client.25501 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 shutdown_connections 2024-09-11T00:24:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.560+0000 7fe69c88c740 1 -- 172.21.15.103:0/2311266777 wait complete. 2024-09-11T00:24:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.561+0000 7fe69c88c740 20 client.25501 trim_cache size 0 max 16384 2024-09-11T00:24:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.582+0000 7fe69c88c740 1 Processor -- start 2024-09-11T00:24:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.582+0000 7fe69c88c740 1 -- start start 2024-09-11T00:24:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3ba1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe69c88c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941b01f0 con 0x55a86d3adb00 2024-09-11T00:24:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe69c88c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941adac0 con 0x55a86d3b3440 2024-09-11T00:24:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe69c88c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941ab910 con 0x55a86d3a8790 2024-09-11T00:24:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3ba1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3ba1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44178/0 (socket says 172.21.15.103:44178) 2024-09-11T00:24:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692a00640 1 -- 172.21.15.103:0/3107290834 learned_addr learned my addr 172.21.15.103:0/3107290834 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692000640 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692000640 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692a00640 1 -- 172.21.15.103:0/3107290834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 msgr2=0x55a86d3b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692a00640 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692a00640 1 -- 172.21.15.103:0/3107290834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 msgr2=0x55a86d3b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.583+0000 7fe692a00640 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.584+0000 7fe692a00640 1 -- 172.21.15.103:0/3107290834 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86d3b2df0 con 0x55a86d3adb00 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.584+0000 7fe692000640 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.584+0000 7fe692a00640 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3ba1c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe684010010 tx=0x7fe6840098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95894d307c5982 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.584+0000 7fe67f400640 1 -- 172.21.15.103:0/3107290834 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68403c020 con 0x55a86d3adb00 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.584+0000 7fe67f400640 1 -- 172.21.15.103:0/3107290834 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684011cb0 con 0x55a86d3adb00 2024-09-11T00:24:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.584+0000 7fe67f400640 1 -- 172.21.15.103:0/3107290834 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684003650 con 0x55a86d3adb00 2024-09-11T00:24:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.585+0000 7fe69c88c740 1 -- 172.21.15.103:0/3107290834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 msgr2=0x55a86d3ba1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.585+0000 7fe69c88c740 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3ba1c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe684010010 tx=0x7fe6840098d0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.585+0000 7fe69c88c740 1 -- 172.21.15.103:0/3107290834 shutdown_connections 2024-09-11T00:24:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.585+0000 7fe69c88c740 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.585+0000 7fe69c88c740 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3ba1c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.585+0000 7fe69c88c740 1 --2- 172.21.15.103:0/3107290834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.585+0000 7fe69c88c740 1 -- 172.21.15.103:0/3107290834 >> 172.21.15.103:0/3107290834 conn(0x55a86d3a38e0 msgr2=0x55a86d3b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.586+0000 7fe69c88c740 1 -- 172.21.15.103:0/3107290834 shutdown_connections 2024-09-11T00:24:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.586+0000 7fe69c88c740 1 -- 172.21.15.103:0/3107290834 wait complete. 2024-09-11T00:24:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.586+0000 7fe69c88c740 1 -- start start 2024-09-11T00:24:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.586+0000 7fe69c88c740 1 Processor -- start 2024-09-11T00:24:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3b7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe69c88c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe69c88c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941abf80 con 0x55a86d3adb00 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe69c88c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941ad3f0 con 0x55a86d3b3440 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe69c88c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941ac650 con 0x55a86d3a8790 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe69c88c740 10 client.0 _ll_register_callbacks cb 0x7fe680009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe692000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe692a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe692a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3b7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44186/0 (socket says 172.21.15.103:44186) 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.587+0000 7fe692a00640 1 -- 172.21.15.103:0/2624633653 learned_addr learned my addr 172.21.15.103:0/2624633653 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.588+0000 7fe692000640 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.588+0000 7fe692a00640 1 -- 172.21.15.103:0/2624633653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 msgr2=0x55a86d3b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.588+0000 7fe692a00640 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.588+0000 7fe692a00640 1 -- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 msgr2=0x55a86d3b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:35.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.588+0000 7fe692a00640 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.588+0000 7fe692a00640 1 -- 172.21.15.103:0/2624633653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86d3b8ce0 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.589+0000 7fe692a00640 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3b7930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe684009670 tx=0x7fe6840056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db1019368fab84f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.589+0000 7fe67cc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.589+0000 7fe67cc00640 1 -- 172.21.15.103:0/2624633653 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68403c030 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.589+0000 7fe67cc00640 1 -- 172.21.15.103:0/2624633653 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe6840037f0 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.589+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe684011cb0 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe67cc00640 1 -- 172.21.15.103:0/2624633653 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe68404f050 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a86d3ba050 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe67cc00640 1 -- 172.21.15.103:0/2624633653 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684011cb0 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe67cc00640 1 -- 172.21.15.103:0/2624633653 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe6840043b0 con 0x55a86d3adb00 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:35.596 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 20 client.15462 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d42ad30 0x55a86d42d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 4 client.15462 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.590+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a86d3190b0 con 0x55a86d42ad30 2024-09-11T00:24:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.591+0000 7fe692000640 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d42ad30 0x55a86d42d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.591+0000 7fe692000640 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d42ad30 0x55a86d42d120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=164d92e424977a6f server_cookie=e81e6255a51ded72 in_seq=0 out_seq=0 2024-09-11T00:24:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.591+0000 7fe67cc00640 10 client.15462 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.595+0000 7fe67cc00640 1 -- 172.21.15.103:0/2624633653 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a86d3190b0 con 0x55a86d42ad30 2024-09-11T00:24:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.595+0000 7fe67cc00640 10 client.15462 handle_command_reply: tid=0 2024-09-11T00:24:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.595+0000 7fe69c88c740 1 client.15462 shutdown 2024-09-11T00:24:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.595+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 msgr2=0x55a86d3b7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3b7930 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe684009670 tx=0x7fe6840056a0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 shutdown_connections 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a86d3b3440 0x55a86d3b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a86d3adb00 0x55a86d3b7930 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a86d42ad30 0x55a86d42d120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 --2- 172.21.15.103:0/2624633653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a86d3a8790 0x55a86d3b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 >> 172.21.15.103:0/2624633653 conn(0x55a86d3a38e0 msgr2=0x55a86d3b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe67cc00640 0 client.15462 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 shutdown_connections 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 1 -- 172.21.15.103:0/2624633653 wait complete. 2024-09-11T00:24:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.596+0000 7fe69c88c740 20 client.15462 trim_cache size 0 max 16384 2024-09-11T00:24:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:24:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:24:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:24:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:24:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6780b5560 msgr2=0x7fe6780b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6780b5560 0x7fe6780b7a20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005ba0 tx=0x7fe68c009a70 comp rx=0 tx=0).stop 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694104490 msgr2=0x7fe694075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694104490 0x7fe694075a80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00eb10 tx=0x7fe68c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 shutdown_connections 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe6780b5560 0x7fe6780b7a20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe694104490 0x7fe694075a80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103ad0 0x7fe694075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 --2- 172.21.15.103:0/433873366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe694102980 0x7fe694076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.597+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 >> 172.21.15.103:0/433873366 conn(0x7fe6940fe240 msgr2=0x7fe694106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.598+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 shutdown_connections 2024-09-11T00:24:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:35.598+0000 7fe699800640 1 -- 172.21.15.103:0/433873366 wait complete. 2024-09-11T00:24:35.651 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:24:35.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:35 smithi117 ceph-mon[35525]: pgmap v443: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-11T00:24:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.408+0000 7f15a8a00640 1 -- 172.21.15.103:0/2922918193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4102c10 msgr2=0x7f15a4102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.408+0000 7f15a8a00640 1 --2- 172.21.15.103:0/2922918193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4102c10 0x7f15a4102ff0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1594009860 tx=0x7f159402a340 comp rx=0 tx=0).stop 2024-09-11T00:24:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.408+0000 7f15a8a00640 1 -- 172.21.15.103:0/2922918193 shutdown_connections 2024-09-11T00:24:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.408+0000 7f15a8a00640 1 --2- 172.21.15.103:0/2922918193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a4108fa0 0x7f15a410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.411 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.408+0000 7f15a8a00640 1 --2- 172.21.15.103:0/2922918193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15a4108620 0x7f15a4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.408+0000 7f15a8a00640 1 --2- 172.21.15.103:0/2922918193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4102c10 0x7f15a4102ff0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.408+0000 7f15a8a00640 1 -- 172.21.15.103:0/2922918193 >> 172.21.15.103:0/2922918193 conn(0x7f15a40fe660 msgr2=0x7f15a4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:36.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.409+0000 7f15a8a00640 1 -- 172.21.15.103:0/2922918193 shutdown_connections 2024-09-11T00:24:36.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.409+0000 7f15a8a00640 1 -- 172.21.15.103:0/2922918193 wait complete. 2024-09-11T00:24:36.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.410+0000 7f15a8a00640 1 Processor -- start 2024-09-11T00:24:36.412 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.410+0000 7f15a8a00640 1 -- start start 2024-09-11T00:24:36.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a8a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a4102c10 0x7f15a410ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a8a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15a4108620 0x7f15a410ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a8a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4108fa0 0x7f15a41a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.414 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a8a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4112c60 con 0x7f15a4108620 2024-09-11T00:24:36.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a8a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4112ae0 con 0x7f15a4102c10 2024-09-11T00:24:36.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a8a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4112de0 con 0x7f15a4108fa0 2024-09-11T00:24:36.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a4102c10 0x7f15a410ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.411+0000 7f15a3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4108fa0 0x7f15a41a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f15a3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4108fa0 0x7f15a41a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:37936/0 (socket says 172.21.15.103:37936) 2024-09-11T00:24:36.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f15a3200640 1 -- 172.21.15.103:0/892142217 learned_addr learned my addr 172.21.15.103:0/892142217 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:36.416 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f159be00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15a4108620 0x7f15a410ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f15a2800640 1 -- 172.21.15.103:0/892142217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4108fa0 msgr2=0x7f15a41a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f15a2800640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4108fa0 0x7f15a41a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f15a2800640 1 -- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15a4108620 msgr2=0x7f15a410ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f15a2800640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15a4108620 0x7f15a410ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f15a2800640 1 -- 172.21.15.103:0/892142217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a41a56b0 con 0x7f15a4102c10 2024-09-11T00:24:36.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.412+0000 7f159be00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15a4108620 0x7f15a410ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:36.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.413+0000 7f15a2800640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a4102c10 0x7f15a410ea00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1594035040 tx=0x7f1594032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:36.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.413+0000 7f159b400640 1 -- 172.21.15.103:0/892142217 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594037040 con 0x7f15a4102c10 2024-09-11T00:24:36.418 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.413+0000 7f159b400640 1 -- 172.21.15.103:0/892142217 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1594030c10 con 0x7f15a4102c10 2024-09-11T00:24:36.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.413+0000 7f159b400640 1 -- 172.21.15.103:0/892142217 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15940035f0 con 0x7f15a4102c10 2024-09-11T00:24:36.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.414+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a41a5940 con 0x7f15a4102c10 2024-09-11T00:24:36.419 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.414+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41ac060 con 0x7f15a4102c10 2024-09-11T00:24:36.421 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.419+0000 7f15ac021740 1 Processor -- start 2024-09-11T00:24:36.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f15ac021740 1 -- start start 2024-09-11T00:24:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f15ac021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8a1d0 0x56123ac8a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f15ac021740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac96540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f15ac021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ac98e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f15ac021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56123ac01290 con 0x56123ac96a80 2024-09-11T00:24:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f15ac021740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56123abf06c0 con 0x56123ac8aaf0 2024-09-11T00:24:36.424 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f15ac021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56123abf44d0 con 0x56123ac8a1d0 2024-09-11T00:24:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.420+0000 7f159b400640 1 -- 172.21.15.103:0/892142217 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1594004050 con 0x7f15a4102c10 2024-09-11T00:24:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f15a2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac96540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f15a2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac96540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51346/0 (socket says 172.21.15.103:51346) 2024-09-11T00:24:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f15a2800640 1 -- 172.21.15.103:0/4134069695 learned_addr learned my addr 172.21.15.103:0/4134069695 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f159be00640 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ac98e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f15a3200640 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8a1d0 0x56123ac8a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f15a2800640 1 -- 172.21.15.103:0/4134069695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8a1d0 msgr2=0x56123ac8a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f15a2800640 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8a1d0 0x56123ac8a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.421+0000 7f15a2800640 1 -- 172.21.15.103:0/4134069695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 msgr2=0x56123ac98e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.422+0000 7f15a2800640 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ac98e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.422+0000 7f15a2800640 1 -- 172.21.15.103:0/4134069695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123ab7d460 con 0x56123ac8aaf0 2024-09-11T00:24:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.422+0000 7f15a3200640 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8a1d0 0x56123ac8a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:24:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.422+0000 7f159be00640 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ac98e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.423+0000 7f15a2800640 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac96540 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1594108f00 tx=0x7f159410d0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9142b7c02152be0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.423+0000 7f1593400640 1 -- 172.21.15.103:0/4134069695 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159410d740 con 0x56123ac8aaf0 2024-09-11T00:24:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.423+0000 7f1593400640 1 -- 172.21.15.103:0/4134069695 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1594112030 con 0x56123ac8aaf0 2024-09-11T00:24:36.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.423+0000 7f1593400640 1 -- 172.21.15.103:0/4134069695 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594105b70 con 0x56123ac8aaf0 2024-09-11T00:24:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.423+0000 7f159b400640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f15880b5490 0x7f15880b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.424+0000 7f15a3200640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f15880b5490 0x7f15880b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.424+0000 7f159b400640 1 -- 172.21.15.103:0/892142217 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1594105060 con 0x7f15a4102c10 2024-09-11T00:24:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.424+0000 7f15ac021740 1 -- 172.21.15.103:0/4134069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 msgr2=0x56123ac96540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.424+0000 7f15ac021740 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac96540 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1594108f00 tx=0x7f159410d0e0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.425+0000 7f15a3200640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f15880b5490 0x7f15880b7950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f159c0049d0 tx=0x7f159c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:36.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.425+0000 7f15ac021740 1 -- 172.21.15.103:0/4134069695 shutdown_connections 2024-09-11T00:24:36.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.425+0000 7f15ac021740 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ac98e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.425+0000 7f15ac021740 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac96540 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.425+0000 7f15ac021740 1 --2- 172.21.15.103:0/4134069695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8a1d0 0x56123ac8a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.425+0000 7f15ac021740 1 -- 172.21.15.103:0/4134069695 >> 172.21.15.103:0/4134069695 conn(0x56123ac88000 msgr2=0x56123ac891a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:36.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.426+0000 7f15ac021740 1 -- 172.21.15.103:0/4134069695 shutdown_connections 2024-09-11T00:24:36.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.426+0000 7f15ac021740 1 -- 172.21.15.103:0/4134069695 wait complete. 2024-09-11T00:24:36.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.426+0000 7f15ac021740 1 -- start start 2024-09-11T00:24:36.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 1 Processor -- start 2024-09-11T00:24:36.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac939e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac93f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac96a80 0x56123ac94460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56123aa51bb0 con 0x56123ac8a1d0 2024-09-11T00:24:36.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56123ab8b6c0 con 0x56123ac8aaf0 2024-09-11T00:24:36.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56123ac994f0 con 0x56123ac96a80 2024-09-11T00:24:36.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.427+0000 7f15ac021740 10 client.0 _ll_register_callbacks cb 0x56123aba6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:36.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f15a3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac96a80 0x56123ac94460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f15a3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac96a80 0x56123ac94460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:37952/0 (socket says 172.21.15.103:37952) 2024-09-11T00:24:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f15a3200640 1 -- 172.21.15.103:0/3913477931 learned_addr learned my addr 172.21.15.103:0/3913477931 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f159be00640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac93f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f15a2800640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac939e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f159be00640 1 -- 172.21.15.103:0/3913477931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac96a80 msgr2=0x56123ac94460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f159be00640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac96a80 0x56123ac94460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.428+0000 7f159be00640 1 -- 172.21.15.103:0/3913477931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 msgr2=0x56123ac939e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f159be00640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac939e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f159be00640 1 -- 172.21.15.103:0/3913477931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123a94eb00 con 0x56123ac8aaf0 2024-09-11T00:24:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f15a2800640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:24:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f159be00640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac93f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f158c009530 tx=0x7f158c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe11bb6e8431770 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f1587e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:24:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f1587e00640 1 -- 172.21.15.103:0/3913477931 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c0120d0 con 0x56123ac8aaf0 2024-09-11T00:24:36.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f1587e00640 1 -- 172.21.15.103:0/3913477931 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f158c010430 con 0x56123ac8aaf0 2024-09-11T00:24:36.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.429+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1594112030 con 0x56123ac8aaf0 2024-09-11T00:24:36.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.430+0000 7f1587e00640 1 -- 172.21.15.103:0/3913477931 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f158c012270 con 0x56123ac8aaf0 2024-09-11T00:24:36.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.430+0000 7f15ac021740 10 client.25487 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:36.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.430+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56123a9c5960 con 0x56123ac8aaf0 2024-09-11T00:24:36.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.430+0000 7f1587e00640 1 -- 172.21.15.103:0/3913477931 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1594112030 con 0x56123ac8aaf0 2024-09-11T00:24:36.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.430+0000 7f1587e00640 1 -- 172.21.15.103:0/3913477931 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f158c01c6c0 con 0x56123ac8aaf0 2024-09-11T00:24:36.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.431+0000 7f15ac021740 10 client.25487 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:36.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.431+0000 7f15ac021740 10 client.25487 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:24:36.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.431+0000 7f15ac021740 20 client.25487 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:36.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.431+0000 7f15ac021740 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123aca6250 0x56123aca86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.431+0000 7f15ac021740 4 client.25487 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:36.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.431+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56123ac031e0 con 0x56123aca6250 2024-09-11T00:24:36.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.432+0000 7f15a2800640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123aca6250 0x56123aca86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.433+0000 7f15a2800640 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123aca6250 0x56123aca86a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f881a1522b09aa7 server_cookie=ca4761c4ffad71ec in_seq=0 out_seq=0 2024-09-11T00:24:36.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.433+0000 7f1587e00640 10 client.25487 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:36.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f1587e00640 1 -- 172.21.15.103:0/3913477931 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56123ac031e0 con 0x56123aca6250 2024-09-11T00:24:36.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f1587e00640 10 client.25487 handle_command_reply: tid=0 2024-09-11T00:24:36.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 client.25487 shutdown 2024-09-11T00:24:36.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 msgr2=0x56123ac93f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac93f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f158c009530 tx=0x7f158c00f790 comp rx=0 tx=0).stop 2024-09-11T00:24:36.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 shutdown_connections 2024-09-11T00:24:36.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123aca6250 0x56123aca86a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac96a80 0x56123ac94460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8aaf0 0x56123ac93f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 --2- 172.21.15.103:0/3913477931 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac939e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 >> 172.21.15.103:0/3913477931 conn(0x56123ac88000 msgr2=0x56123ac8dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:36.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.434+0000 7f1587e00640 0 client.25487 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:36.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.435+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 shutdown_connections 2024-09-11T00:24:36.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.435+0000 7f15ac021740 1 -- 172.21.15.103:0/3913477931 wait complete. 2024-09-11T00:24:36.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.435+0000 7f15ac021740 20 client.25487 trim_cache size 0 max 16384 2024-09-11T00:24:36.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.469+0000 7f15ac021740 1 Processor -- start 2024-09-11T00:24:36.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.469+0000 7f15ac021740 1 -- start start 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15ac021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac8ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15ac021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac95720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15ac021740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac96a80 0x56123ac90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15ac021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b8740 con 0x56123ac8a1d0 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15ac021740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b6010 con 0x56123ac96a80 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15ac021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b3e60 con 0x56123ac8aaf0 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15a2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac8ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f159be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac95720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f159be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac95720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:37960/0 (socket says 172.21.15.103:37960) 2024-09-11T00:24:36.474 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f15a2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac8ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44236/0 (socket says 172.21.15.103:44236) 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.470+0000 7f159be00640 1 -- 172.21.15.103:0/1909420381 learned_addr learned my addr 172.21.15.103:0/1909420381 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.471+0000 7f15a3200640 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac96a80 0x56123ac90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.471+0000 7f159be00640 1 -- 172.21.15.103:0/1909420381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac96a80 msgr2=0x56123ac90bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.471+0000 7f159be00640 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac96a80 0x56123ac90bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.471+0000 7f159be00640 1 -- 172.21.15.103:0/1909420381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 msgr2=0x56123ac8ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.471+0000 7f159be00640 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac8ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.471+0000 7f159be00640 1 -- 172.21.15.103:0/1909420381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123ac92940 con 0x56123ac8aaf0 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.471+0000 7f159be00640 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac95720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f158c01cd20 tx=0x7f158c009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db489745888b62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.472+0000 7f1592a00640 1 -- 172.21.15.103:0/1909420381 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c02f070 con 0x56123ac8aaf0 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.472+0000 7f1592a00640 1 -- 172.21.15.103:0/1909420381 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158c01c8d0 con 0x56123ac8aaf0 2024-09-11T00:24:36.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.472+0000 7f1592a00640 1 -- 172.21.15.103:0/1909420381 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c0048c0 con 0x56123ac8aaf0 2024-09-11T00:24:36.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 -- 172.21.15.103:0/1909420381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 msgr2=0x56123ac95720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac95720 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f158c01cd20 tx=0x7f158c009fd0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 -- 172.21.15.103:0/1909420381 shutdown_connections 2024-09-11T00:24:36.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac96a80 0x56123ac90bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac95720 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 --2- 172.21.15.103:0/1909420381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac8a1d0 0x56123ac8ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 -- 172.21.15.103:0/1909420381 >> 172.21.15.103:0/1909420381 conn(0x56123ac88000 msgr2=0x56123ac95010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.473+0000 7f15ac021740 1 -- 172.21.15.103:0/1909420381 shutdown_connections 2024-09-11T00:24:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.474+0000 7f15ac021740 1 -- 172.21.15.103:0/1909420381 wait complete. 2024-09-11T00:24:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.474+0000 7f15ac021740 1 -- start start 2024-09-11T00:24:36.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.474+0000 7f15ac021740 1 Processor -- start 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.475+0000 7f15ac021740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8a1d0 0x56123ac9ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.475+0000 7f15ac021740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac9bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.475+0000 7f15ac021740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ad131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.475+0000 7f15ac021740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b44d0 con 0x56123ac96a80 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.475+0000 7f15ac021740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b5940 con 0x56123ac8a1d0 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.475+0000 7f15ac021740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15a41b4ba0 con 0x56123ac8aaf0 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.475+0000 7f15ac021740 10 client.0 _ll_register_callbacks cb 0x7f158c009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8a1d0 0x56123ac9ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8a1d0 0x56123ac9ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51370/0 (socket says 172.21.15.103:51370) 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 -- 172.21.15.103:0/1840797794 learned_addr learned my addr 172.21.15.103:0/1840797794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:36.478 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f159be00640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac9bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a3200640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ad131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 -- 172.21.15.103:0/1840797794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 msgr2=0x56123ac9bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac9bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 -- 172.21.15.103:0/1840797794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 msgr2=0x56123ad131c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ad131c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a2800640 1 -- 172.21.15.103:0/1840797794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123ac8a920 con 0x56123ac8a1d0 2024-09-11T00:24:36.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.476+0000 7f15a3200640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ad131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:24:36.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.477+0000 7f15a2800640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8a1d0 0x56123ac9ba80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15940cb130 tx=0x7f1594030fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135e39c3d0cc542d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:36.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.477+0000 7f1587e00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:24:36.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.477+0000 7f1587e00640 1 -- 172.21.15.103:0/1840797794 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15940c7a80 con 0x56123ac8a1d0 2024-09-11T00:24:36.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.477+0000 7f1587e00640 1 -- 172.21.15.103:0/1840797794 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f1594108800 con 0x56123ac8a1d0 2024-09-11T00:24:36.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.477+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f158c01c8d0 con 0x56123ac8a1d0 2024-09-11T00:24:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.481+0000 7f1587e00640 1 -- 172.21.15.103:0/1840797794 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1594105490 con 0x56123ac8a1d0 2024-09-11T00:24:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.481+0000 7f15ac021740 10 client.25493 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:36.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.482+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56123ad14740 con 0x56123ac8a1d0 2024-09-11T00:24:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.482+0000 7f1587e00640 1 -- 172.21.15.103:0/1840797794 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f158c01c8d0 con 0x56123ac8a1d0 2024-09-11T00:24:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.482+0000 7f1587e00640 1 -- 172.21.15.103:0/1840797794 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f15940c76a0 con 0x56123ac8a1d0 2024-09-11T00:24:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.483+0000 7f15ac021740 10 client.25493 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:36.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.483+0000 7f15ac021740 10 client.25493 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:24:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.483+0000 7f15ac021740 20 client.25493 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.483+0000 7f15ac021740 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123ad14c30 0x56123ad17050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.483+0000 7f15ac021740 4 client.25493 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "21e61e96", "release": true, "await": true}] 2024-09-11T00:24:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.483+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "set_id": "21e61e96", "release": true, "await": true}) -- 0x56123abfec90 con 0x56123ad14c30 2024-09-11T00:24:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.483+0000 7f159be00640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123ad14c30 0x56123ad17050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.484+0000 7f159be00640 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123ad14c30 0x56123ad17050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a3fe4e048cd413 server_cookie=6c7368c3e7fa4c25 in_seq=0 out_seq=0 2024-09-11T00:24:36.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.484+0000 7f1587e00640 10 client.25493 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.485+0000 7f1587e00640 1 -- 172.21.15.103:0/1840797794 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56123abfec90 con 0x56123ad14c30 2024-09-11T00:24:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.485+0000 7f1587e00640 10 client.25493 handle_command_reply: tid=0 2024-09-11T00:24:36.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.485+0000 7f15ac021740 1 client.25493 shutdown 2024-09-11T00:24:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.486+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8a1d0 msgr2=0x56123ac9ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.486+0000 7f15ac021740 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8a1d0 0x56123ac9ba80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f15940cb130 tx=0x7f1594030fc0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.486+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 shutdown_connections 2024-09-11T00:24:36.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.486+0000 7f15ac021740 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56123ac96a80 0x56123ad131c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.486+0000 7f15ac021740 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56123ad14c30 0x56123ad17050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.487+0000 7f15ac021740 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56123ac8aaf0 0x56123ac9bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.487+0000 7f1587e00640 0 client.25493 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:36.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.487+0000 7f15ac021740 1 --2- 172.21.15.103:0/1840797794 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x56123ac8a1d0 0x56123ac9ba80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.487+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 >> 172.21.15.103:0/1840797794 conn(0x56123ac88000 msgr2=0x56123ac95830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.487+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 shutdown_connections 2024-09-11T00:24:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.488+0000 7f15ac021740 1 -- 172.21.15.103:0/1840797794 wait complete. 2024-09-11T00:24:36.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.488+0000 7f15ac021740 20 client.25493 trim_cache size 0 max 16384 2024-09-11T00:24:36.490 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 27, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 4, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "21e61e96": { 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 4, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 145.2, 2024-09-11T00:24:36.491 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:24:36.492 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f15880b5490 msgr2=0x7f15880b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f15880b5490 0x7f15880b7950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f159c0049d0 tx=0x7f159c009500 comp rx=0 tx=0).stop 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a4102c10 msgr2=0x7f15a410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a4102c10 0x7f15a410ea00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1594035040 tx=0x7f1594032d50 comp rx=0 tx=0).stop 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 shutdown_connections 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f15880b5490 0x7f15880b7950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15a4108fa0 0x7f15a41a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15a4108620 0x7f15a410ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 --2- 172.21.15.103:0/892142217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15a4102c10 0x7f15a410ea00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.489+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 >> 172.21.15.103:0/892142217 conn(0x7f15a40fe660 msgr2=0x7f15a40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.490+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 shutdown_connections 2024-09-11T00:24:36.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:24:36.490+0000 7f15a8a00640 1 -- 172.21.15.103:0/892142217 wait complete. 2024-09-11T00:24:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:36 smithi183 ceph-mon[35934]: pgmap v444: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T00:24:36.909 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:36 smithi103 ceph-mon[30578]: pgmap v444: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T00:24:36.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:36 smithi117 ceph-mon[35525]: pgmap v444: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T00:24:37.494 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '21e61e96', total seconds elapsed: 30.1 2024-09-11T00:24:38.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:38 smithi183 ceph-mon[35934]: pgmap v445: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T00:24:38.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:38 smithi103 ceph-mon[30578]: pgmap v445: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T00:24:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:38 smithi117 ceph-mon[35525]: pgmap v445: 129 pgs: 129 active+clean; 668 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-11T00:24:39.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:39 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:24:39.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:39 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:24:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:39 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:24:40.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:24:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:24:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:24:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:40 smithi183 ceph-mon[35934]: pgmap v446: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-11T00:24:40.847 INFO:tasks.workunit.client.0.smithi103.stderr:Updating files: 63% (7699/12088) Updating files: 64% (7737/12088) Updating files: 65% (7858/12088) Updating files: 66% (7979/12088) Updating files: 67% (8099/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-11T00:24:40.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:24:40.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:24:40.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:24:40.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:40 smithi103 ceph-mon[30578]: pgmap v446: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-11T00:24:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:24:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:24:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:24:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:40 smithi117 ceph-mon[35525]: pgmap v446: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-11T00:24:42.196 INFO:tasks.workunit.client.0.smithi103.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr: git switch -c 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr:Or undo this operation with: 2024-09-11T00:24:42.197 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:24:42.198 INFO:tasks.workunit.client.0.smithi103.stderr: git switch - 2024-09-11T00:24:42.198 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:24:42.198 INFO:tasks.workunit.client.0.smithi103.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T00:24:42.198 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:24:42.198 INFO:tasks.workunit.client.0.smithi103.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-11T00:24:42.209 DEBUG:teuthology.orchestra.run.smithi103:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T00:24:42.249 INFO:tasks.workunit.client.0.smithi103.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T00:24:42.254 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T00:24:42.254 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T00:24:42.739 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:42 smithi117 ceph-mon[35525]: pgmap v447: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-11T00:24:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:42 smithi183 ceph-mon[35934]: pgmap v447: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-11T00:24:42.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:42 smithi103 ceph-mon[30578]: pgmap v447: 129 pgs: 129 active+clean; 669 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-11T00:24:43.021 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T00:24:43.119 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T00:24:43.185 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T00:24:43.189 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T00:24:43.189 INFO:tasks.workunit.client.0.smithi103.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T00:24:43.254 INFO:tasks.workunit.client.0.smithi103.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T00:24:43.261 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:24:43.261 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T00:24:43.338 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-09-11T00:24:43.340 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-09-11T00:24:43.340 DEBUG:teuthology.orchestra.run.smithi103:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-09-11T00:24:43.428 INFO:tasks.workunit.client.0.smithi103.stderr:+ set -e 2024-09-11T00:24:43.428 INFO:tasks.workunit.client.0.smithi103.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-09-11T00:24:43.432 INFO:tasks.workunit.client.0.smithi103.stderr:Cloning into 'xfstests-dev'... 2024-09-11T00:24:44.767 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:44 smithi103 ceph-mon[30578]: pgmap v448: 129 pgs: 129 active+clean; 751 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7.2 KiB/s wr, 7 op/s 2024-09-11T00:24:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:44 smithi183 ceph-mon[35934]: pgmap v448: 129 pgs: 129 active+clean; 751 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7.2 KiB/s wr, 7 op/s 2024-09-11T00:24:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:44 smithi117 ceph-mon[35525]: pgmap v448: 129 pgs: 129 active+clean; 751 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7.2 KiB/s wr, 7 op/s 2024-09-11T00:24:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:46 smithi183 ceph-mon[35934]: pgmap v449: 129 pgs: 129 active+clean; 751 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2024-09-11T00:24:46.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:46 smithi103 ceph-mon[30578]: pgmap v449: 129 pgs: 129 active+clean; 751 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2024-09-11T00:24:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:46 smithi117 ceph-mon[35525]: pgmap v449: 129 pgs: 129 active+clean; 751 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7.2 KiB/s wr, 6 op/s 2024-09-11T00:24:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 -- 172.21.15.103:0/128236437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff81062a0 msgr2=0x7efff8110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 --2- 172.21.15.103:0/128236437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff81062a0 0x7efff8110f80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efff0009900 tx=0x7efff002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 -- 172.21.15.103:0/128236437 shutdown_connections 2024-09-11T00:24:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 --2- 172.21.15.103:0/128236437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff81062a0 0x7efff8110f80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 --2- 172.21.15.103:0/128236437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 0x7efff8075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 --2- 172.21.15.103:0/128236437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff8076f90 0x7efff80755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 -- 172.21.15.103:0/128236437 >> 172.21.15.103:0/128236437 conn(0x7efff80fe030 msgr2=0x7efff8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 -- 172.21.15.103:0/128236437 shutdown_connections 2024-09-11T00:24:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.728+0000 7efffd600640 1 -- 172.21.15.103:0/128236437 wait complete. 2024-09-11T00:24:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.729+0000 7efffd600640 1 Processor -- start 2024-09-11T00:24:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.729+0000 7efffd600640 1 -- start start 2024-09-11T00:24:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efffd600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 0x7efff819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efffd600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff8076f90 0x7efff819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efffd600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff81a0390 0x7efff81a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efffd600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efff8114a10 con 0x7efff8075b20 2024-09-11T00:24:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efffd600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efff8114890 con 0x7efff81a0390 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efffd600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efff8114b90 con 0x7efff8076f90 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efff7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 0x7efff819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efff7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 0x7efff819f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53836/0 (socket says 172.21.15.103:53836) 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efff7e00640 1 -- 172.21.15.103:0/1017174734 learned_addr learned my addr 172.21.15.103:0/1017174734 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7effef400640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff8076f90 0x7efff819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efffcc00640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff81a0390 0x7efff81a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efff7e00640 1 -- 172.21.15.103:0/1017174734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff8076f90 msgr2=0x7efff819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.730+0000 7efff7e00640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff8076f90 0x7efff819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efff7e00640 1 -- 172.21.15.103:0/1017174734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff81a0390 msgr2=0x7efff81a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efff7e00640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff81a0390 0x7efff81a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efff7e00640 1 -- 172.21.15.103:0/1017174734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff81a1040 con 0x7efff8075b20 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7effef400640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff8076f90 0x7efff819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efff7e00640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 0x7efff819f110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effe800eb90 tx=0x7effe800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efff5600640 1 -- 172.21.15.103:0/1017174734 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8004d20 con 0x7efff8075b20 2024-09-11T00:24:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efff5600640 1 -- 172.21.15.103:0/1017174734 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe800ee90 con 0x7efff8075b20 2024-09-11T00:24:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff81a7430 con 0x7efff8075b20 2024-09-11T00:24:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efff5600640 1 -- 172.21.15.103:0/1017174734 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8010670 con 0x7efff8075b20 2024-09-11T00:24:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.731+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a78d0 con 0x7efff8075b20 2024-09-11T00:24:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 Processor -- start 2024-09-11T00:24:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7efff5600640 1 -- 172.21.15.103:0/1017174734 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7effe8005000 con 0x7efff8075b20 2024-09-11T00:24:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 -- start start 2024-09-11T00:24:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b17d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b1a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b1ace0 0x55a967b1d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a967a886b0 con 0x55a967b15970 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a967a77ae0 con 0x55a967b182a0 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.735+0000 7f0001a5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a967a7b8f0 con 0x55a967b1ace0 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b17d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efff7e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b1a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b17d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53850/0 (socket says 172.21.15.103:53850) 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 -- 172.21.15.103:0/522542204 learned_addr learned my addr 172.21.15.103:0/522542204 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7effef400640 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b1ace0 0x55a967b1d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 -- 172.21.15.103:0/522542204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b1ace0 msgr2=0x55a967b1d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b1ace0 0x55a967b1d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 -- 172.21.15.103:0/522542204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 msgr2=0x55a967b1a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b1a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 -- 172.21.15.103:0/522542204 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a967a44780 con 0x55a967b15970 2024-09-11T00:24:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7effef400640 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b1ace0 0x55a967b1d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efff7e00640 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b1a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:48.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.736+0000 7efffcc00640 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b17d60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efff0033000 tx=0x7efff0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb001e451d8a2431 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7effed200640 1 -- 172.21.15.103:0/522542204 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0035030 con 0x55a967b15970 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7effed200640 1 -- 172.21.15.103:0/522542204 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0040070 con 0x55a967b15970 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7effed200640 1 -- 172.21.15.103:0/522542204 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0003930 con 0x55a967b15970 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7efff5600640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effdc0b52e0 0x7effdc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7efff5600640 1 -- 172.21.15.103:0/1017174734 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7effe80e65d0 con 0x7efff8075b20 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7efff7e00640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effdc0b52e0 0x7effdc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7f0001a5c740 1 -- 172.21.15.103:0/522542204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 msgr2=0x55a967b17d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.737+0000 7f0001a5c740 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b17d60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efff0033000 tx=0x7efff0005dd0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 -- 172.21.15.103:0/522542204 shutdown_connections 2024-09-11T00:24:48.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b1ace0 0x55a967b1d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b1a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 --2- 172.21.15.103:0/522542204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b17d60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 -- 172.21.15.103:0/522542204 >> 172.21.15.103:0/522542204 conn(0x55a967b10ac0 msgr2=0x55a967b11850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7efff7e00640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effdc0b52e0 0x7effdc0b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7effe80ac930 tx=0x7effe80e9f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 -- 172.21.15.103:0/522542204 shutdown_connections 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 -- 172.21.15.103:0/522542204 wait complete. 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 -- start start 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.738+0000 7f0001a5c740 1 Processor -- start 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b202e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7f0001a5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a9679afab0 con 0x55a967b15970 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7f0001a5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a967a1e570 con 0x55a967b1ace0 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7f0001a5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a967b1dc60 con 0x55a967b182a0 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7f0001a5c740 10 client.0 _ll_register_callbacks cb 0x55a967a2d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7efffcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7efffcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53854/0 (socket says 172.21.15.103:53854) 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7efffcc00640 1 -- 172.21.15.103:0/4289823725 learned_addr learned my addr 172.21.15.103:0/4289823725 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7effef400640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.739+0000 7efffcc00640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effef400640 1 -- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 msgr2=0x55a967b202e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effef400640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b202e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effef400640 1 -- 172.21.15.103:0/4289823725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 msgr2=0x55a967b1f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effef400640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effef400640 1 -- 172.21.15.103:0/4289823725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9677d9790 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7efffcc00640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effef400640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1f710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effe0012960 tx=0x7effe0012e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4138d1e86dd7ae8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effe5600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effe5600640 1 -- 172.21.15.103:0/4289823725 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe000fa00 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff0040070 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effe5600640 1 -- 172.21.15.103:0/4289823725 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7effe000fba0 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7effe5600640 1 -- 172.21.15.103:0/4289823725 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effe0025050 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7f0001a5c740 10 client.25534 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.740+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a967852620 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7effe5600640 1 -- 172.21.15.103:0/4289823725 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0040070 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7effe5600640 1 -- 172.21.15.103:0/4289823725 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7effe0010430 con 0x55a967b182a0 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7f0001a5c740 10 client.25534 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7f0001a5c740 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7f0001a5c740 20 client.25534 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b2aa30 0x55a967b2ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7f0001a5c740 4 client.25534 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:24:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a967a8a600 con 0x55a967b2aa30 2024-09-11T00:24:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.741+0000 7efffcc00640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b2aa30 0x55a967b2ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.742+0000 7efffcc00640 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b2aa30 0x55a967b2ce50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2a054d22dda987 server_cookie=2010dc49e599edef in_seq=0 out_seq=0 2024-09-11T00:24:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.742+0000 7effe5600640 10 client.25534 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7effe5600640 1 -- 172.21.15.103:0/4289823725 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a967a8a600 con 0x55a967b2aa30 2024-09-11T00:24:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7effe5600640 10 client.25534 handle_command_reply: tid=0 2024-09-11T00:24:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 client.25534 shutdown 2024-09-11T00:24:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 msgr2=0x55a967b1f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1f710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7effe0012960 tx=0x7effe0012e20 comp rx=0 tx=0).stop 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 shutdown_connections 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b202e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b2aa30 0x55a967b2ce50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1f710 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4289823725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7effe5600640 0 client.25534 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 >> 172.21.15.103:0/4289823725 conn(0x55a967b10ac0 msgr2=0x55a967b0fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 shutdown_connections 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 1 -- 172.21.15.103:0/4289823725 wait complete. 2024-09-11T00:24:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.743+0000 7f0001a5c740 20 client.25534 trim_cache size 0 max 16384 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.758+0000 7f0001a5c740 1 Processor -- start 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.758+0000 7f0001a5c740 1 -- start start 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.758+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.758+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7f0001a5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b4230 con 0x55a967b15970 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7f0001a5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b1b00 con 0x55a967b1ace0 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7f0001a5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efff81af950 con 0x55a967b182a0 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7efffcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7efffcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53866/0 (socket says 172.21.15.103:53866) 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7efffcc00640 1 -- 172.21.15.103:0/4007422096 learned_addr learned my addr 172.21.15.103:0/4007422096 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:48.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7effef400640 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7efffcc00640 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7effef400640 1 -- 172.21.15.103:0/4007422096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 msgr2=0x55a967b128f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7effef400640 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b128f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7effef400640 1 -- 172.21.15.103:0/4007422096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 msgr2=0x55a967b1b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7effef400640 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7effef400640 1 -- 172.21.15.103:0/4007422096 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a967b1eb70 con 0x55a967b182a0 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.759+0000 7efffcc00640 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.760+0000 7effef400640 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1b600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effe0010800 tx=0x7effe00123a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=845a61f0024072ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.760+0000 7effe7e00640 1 -- 172.21.15.103:0/4007422096 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0010430 con 0x55a967b182a0 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.760+0000 7effe7e00640 1 -- 172.21.15.103:0/4007422096 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe00105d0 con 0x55a967b182a0 2024-09-11T00:24:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.760+0000 7effe7e00640 1 -- 172.21.15.103:0/4007422096 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0005260 con 0x55a967b182a0 2024-09-11T00:24:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.760+0000 7f0001a5c740 1 -- 172.21.15.103:0/4007422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 msgr2=0x55a967b1b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.760+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1b600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effe0010800 tx=0x7effe00123a0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- 172.21.15.103:0/4007422096 shutdown_connections 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b1ace0 0x55a967b128f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b182a0 0x55a967b1b600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 --2- 172.21.15.103:0/4007422096 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b15970 0x55a967b1b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- 172.21.15.103:0/4007422096 >> 172.21.15.103:0/4007422096 conn(0x55a967b10ac0 msgr2=0x55a967b14e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- 172.21.15.103:0/4007422096 shutdown_connections 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- 172.21.15.103:0/4007422096 wait complete. 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- start start 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 Processor -- start 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b15970 0x55a967b25c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b1ace0 0x55a967b26710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efff81affc0 con 0x55a967b1ace0 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b1430 con 0x55a967b182a0 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.761+0000 7f0001a5c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efff81b0690 con 0x55a967b15970 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7f0001a5c740 10 client.0 _ll_register_callbacks cb 0x7effe0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b261d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40160/0 (socket says 172.21.15.103:40160) 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 -- 172.21.15.103:0/3094740553 learned_addr learned my addr 172.21.15.103:0/3094740553 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7efffcc00640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b1ace0 0x55a967b26710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7efffcc00640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b15970 0x55a967b25c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 -- 172.21.15.103:0/3094740553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b15970 msgr2=0x55a967b25c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b15970 0x55a967b25c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 -- 172.21.15.103:0/3094740553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b1ace0 msgr2=0x55a967b26710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:24:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b1ace0 0x55a967b26710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 -- 172.21.15.103:0/3094740553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a967b24930 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7efffcc00640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b1ace0 0x55a967b26710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effef400640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b261d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effe0012930 tx=0x7effe0012490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4eae389de57d899 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.762+0000 7effe5600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.763+0000 7effe5600640 1 -- 172.21.15.103:0/3094740553 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0010430 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.763+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effe00105d0 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.763+0000 7effe5600640 1 -- 172.21.15.103:0/3094740553 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7effe0018040 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.763+0000 7effe5600640 1 -- 172.21.15.103:0/3094740553 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7effe0010bb0 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.763+0000 7f0001a5c740 10 client.25511 fetch_fsmap learned FSMap version 27 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.763+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a967b97da0 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.763+0000 7effe5600640 1 -- 172.21.15.103:0/3094740553 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effe00105d0 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7effe5600640 1 -- 172.21.15.103:0/3094740553 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7effe0003bd0 con 0x55a967b182a0 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7f0001a5c740 10 client.25511 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T00:24:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7f0001a5c740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:24:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7f0001a5c740 20 client.25511 populate_metadata read hostname 'smithi103' 2024-09-11T00:24:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7f0001a5c740 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b986f0 0x55a967b9ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:24:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7f0001a5c740 4 client.25511 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:24:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a967a860b0 con 0x55a967b986f0 2024-09-11T00:24:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7efffcc00640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b986f0 0x55a967b9ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:24:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7efffcc00640 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b986f0 0x55a967b9ab10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8dfe3a8025c71a server_cookie=43c8f1c46bce7ac8 in_seq=0 out_seq=0 2024-09-11T00:24:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:48.764+0000 7effe5600640 10 client.25511 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:48 smithi183 ceph-mon[35934]: pgmap v450: 129 pgs: 129 active+clean; 4.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 351 KiB/s wr, 6 op/s 2024-09-11T00:24:48.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:48 smithi103 ceph-mon[30578]: pgmap v450: 129 pgs: 129 active+clean; 4.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 351 KiB/s wr, 6 op/s 2024-09-11T00:24:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:48 smithi117 ceph-mon[35525]: pgmap v450: 129 pgs: 129 active+clean; 4.7 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 351 KiB/s wr, 6 op/s 2024-09-11T00:24:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.069+0000 7effe5600640 1 -- 172.21.15.103:0/3094740553 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a967a860b0 con 0x55a967b986f0 2024-09-11T00:24:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.069+0000 7effe5600640 10 client.25511 handle_command_reply: tid=0 2024-09-11T00:24:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.069+0000 7f0001a5c740 1 client.25511 shutdown 2024-09-11T00:24:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 msgr2=0x55a967b261d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b261d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7effe0012930 tx=0x7effe0012490 comp rx=0 tx=0).stop 2024-09-11T00:24:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 shutdown_connections 2024-09-11T00:24:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a967b1ace0 0x55a967b26710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a967b182a0 0x55a967b261d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a967b986f0 0x55a967b9ab10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 --2- 172.21.15.103:0/3094740553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a967b15970 0x55a967b25c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 >> 172.21.15.103:0/3094740553 conn(0x55a967b10ac0 msgr2=0x55a967b14d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7effe5600640 0 client.25511 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 shutdown_connections 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 1 -- 172.21.15.103:0/3094740553 wait complete. 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.070+0000 7f0001a5c740 20 client.25511 trim_cache size 0 max 16384 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:24:49.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:24:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effdc0b52e0 msgr2=0x7effdc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effdc0b52e0 0x7effdc0b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7effe80ac930 tx=0x7effe80e9f90 comp rx=0 tx=0).stop 2024-09-11T00:24:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 msgr2=0x7efff819f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:24:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 0x7efff819f110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7effe800eb90 tx=0x7effe800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 shutdown_connections 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7effdc0b52e0 0x7effdc0b77a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efff81a0390 0x7efff81a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efff8076f90 0x7efff819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 --2- 172.21.15.103:0/1017174734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efff8075b20 0x7efff819f110 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 >> 172.21.15.103:0/1017174734 conn(0x7efff80fe030 msgr2=0x7efff80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 shutdown_connections 2024-09-11T00:24:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:24:49.071+0000 7efffd600640 1 -- 172.21.15.103:0/1017174734 wait complete. 2024-09-11T00:24:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:50 smithi183 ceph-mon[35934]: pgmap v451: 129 pgs: 129 active+clean; 10 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 837 KiB/s wr, 20 op/s 2024-09-11T00:24:50.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:50 smithi103 ceph-mon[30578]: pgmap v451: 129 pgs: 129 active+clean; 10 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 837 KiB/s wr, 20 op/s 2024-09-11T00:24:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:50 smithi117 ceph-mon[35525]: pgmap v451: 129 pgs: 129 active+clean; 10 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 837 KiB/s wr, 20 op/s 2024-09-11T00:24:52.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:52 smithi183 ceph-mon[35934]: pgmap v452: 129 pgs: 129 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-09-11T00:24:52.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:52 smithi103 ceph-mon[30578]: pgmap v452: 129 pgs: 129 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-09-11T00:24:52.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:52 smithi117 ceph-mon[35525]: pgmap v452: 129 pgs: 129 active+clean; 15 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-09-11T00:24:54.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:54 smithi103 ceph-mon[30578]: pgmap v453: 129 pgs: 129 active+clean; 16 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-11T00:24:54.784 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:54 smithi183 ceph-mon[35934]: pgmap v453: 129 pgs: 129 active+clean; 16 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-11T00:24:54.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:54 smithi117 ceph-mon[35525]: pgmap v453: 129 pgs: 129 active+clean; 16 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 122 op/s 2024-09-11T00:24:56.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:56 smithi183 ceph-mon[35934]: pgmap v454: 129 pgs: 129 active+clean; 16 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-11T00:24:56.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:56 smithi103 ceph-mon[30578]: pgmap v454: 129 pgs: 129 active+clean; 16 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-11T00:24:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:56 smithi117 ceph-mon[35525]: pgmap v454: 129 pgs: 129 active+clean; 16 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-11T00:24:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:24:58 smithi183 ceph-mon[35934]: pgmap v455: 129 pgs: 129 active+clean; 16 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-11T00:24:58.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:24:58 smithi103 ceph-mon[30578]: pgmap v455: 129 pgs: 129 active+clean; 16 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-11T00:24:58.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:24:58 smithi117 ceph-mon[35525]: pgmap v455: 129 pgs: 129 active+clean; 16 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-11T00:25:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:00 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:00.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:00 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:00.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:00 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:01.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:01 smithi183 ceph-mon[35934]: pgmap v456: 129 pgs: 129 active+clean; 20 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-09-11T00:25:01.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:01 smithi103 ceph-mon[30578]: pgmap v456: 129 pgs: 129 active+clean; 20 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-09-11T00:25:01.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:01 smithi117 ceph-mon[35525]: pgmap v456: 129 pgs: 129 active+clean; 20 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 1.4 MiB/s wr, 138 op/s 2024-09-11T00:25:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 -- 172.21.15.103:0/2635072309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 msgr2=0x7fe11c0755e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 --2- 172.21.15.103:0/2635072309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 0x7fe11c0755e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe110009900 tx=0x7fe11002a400 comp rx=0 tx=0).stop 2024-09-11T00:25:02.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 -- 172.21.15.103:0/2635072309 shutdown_connections 2024-09-11T00:25:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 --2- 172.21.15.103:0/2635072309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe11c10a6b0 0x7fe11c10cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 --2- 172.21.15.103:0/2635072309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 0x7fe11c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 --2- 172.21.15.103:0/2635072309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 0x7fe11c0755e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 -- 172.21.15.103:0/2635072309 >> 172.21.15.103:0/2635072309 conn(0x7fe11c0fe030 msgr2=0x7fe11c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 -- 172.21.15.103:0/2635072309 shutdown_connections 2024-09-11T00:25:02.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.197+0000 7fe122000640 1 -- 172.21.15.103:0/2635072309 wait complete. 2024-09-11T00:25:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.198+0000 7fe122000640 1 Processor -- start 2024-09-11T00:25:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.198+0000 7fe122000640 1 -- start start 2024-09-11T00:25:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe122000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 0x7fe11c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe122000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 0x7fe11c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe122000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe11c10a6b0 0x7fe11c19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe122000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1105e0 con 0x7fe11c10a6b0 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe122000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c110460 con 0x7fe11c076f90 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe122000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c110760 con 0x7fe11c075b20 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe120c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 0x7fe11c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe121600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe11c10a6b0 0x7fe11c19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe11be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 0x7fe11c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe120c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 0x7fe11c19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:46710/0 (socket says 172.21.15.103:46710) 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe120c00640 1 -- 172.21.15.103:0/1107077697 learned_addr learned my addr 172.21.15.103:0/1107077697 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe11be00640 1 -- 172.21.15.103:0/1107077697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 msgr2=0x7fe11c19ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe11be00640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 0x7fe11c19ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.203 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe11be00640 1 -- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe11c10a6b0 msgr2=0x7fe11c19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe11be00640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe11c10a6b0 0x7fe11c19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe11be00640 1 -- 172.21.15.103:0/1107077697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c19c8d0 con 0x7fe11c076f90 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.199+0000 7fe120c00640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 0x7fe11c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.200+0000 7fe11be00640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 0x7fe11c19b3e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe10800e9f0 tx=0x7fe10800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.200+0000 7fe119600640 1 -- 172.21.15.103:0/1107077697 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10800ccc0 con 0x7fe11c076f90 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.200+0000 7fe119600640 1 -- 172.21.15.103:0/1107077697 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1080108e0 con 0x7fe11c076f90 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.200+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c19cbc0 con 0x7fe11c076f90 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.200+0000 7fe119600640 1 -- 172.21.15.103:0/1107077697 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108018840 con 0x7fe11c076f90 2024-09-11T00:25:02.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.200+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c1a3290 con 0x7fe11c076f90 2024-09-11T00:25:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 Processor -- start 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 -- start start 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faacf9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacfc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfcce0 0x558faacff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558faac6a6b0 con 0x558faacfa2a0 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558faac59ae0 con 0x558faacf7970 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe12663b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558faac5d8f0 con 0x558faacfcce0 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe121600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faacf9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe121600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faacf9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40330/0 (socket says 172.21.15.103:40330) 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe121600640 1 -- 172.21.15.103:0/3000475623 learned_addr learned my addr 172.21.15.103:0/3000475623 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe120c00640 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacfc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.204+0000 7fe119600640 1 -- 172.21.15.103:0/1107077697 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe108010430 con 0x7fe11c076f90 2024-09-11T00:25:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe121600640 1 -- 172.21.15.103:0/3000475623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfcce0 msgr2=0x558faacff1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:25:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe121600640 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfcce0 0x558faacff1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe121600640 1 -- 172.21.15.103:0/3000475623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 msgr2=0x558faacfc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe121600640 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacfc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe121600640 1 -- 172.21.15.103:0/3000475623 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558faac26780 con 0x558faacf7970 2024-09-11T00:25:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe121600640 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faacf9d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe114009fa0 tx=0x7fe1140118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be40ad36177128d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe10dc00640 1 -- 172.21.15.103:0/3000475623 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114004d20 con 0x558faacf7970 2024-09-11T00:25:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe10dc00640 1 -- 172.21.15.103:0/3000475623 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe114004050 con 0x558faacf7970 2024-09-11T00:25:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.205+0000 7fe10dc00640 1 -- 172.21.15.103:0/3000475623 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114005280 con 0x558faacf7970 2024-09-11T00:25:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 -- 172.21.15.103:0/3000475623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 msgr2=0x558faacf9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faacf9d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe114009fa0 tx=0x7fe1140118d0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 -- 172.21.15.103:0/3000475623 shutdown_connections 2024-09-11T00:25:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfcce0 0x558faacff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacfc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 --2- 172.21.15.103:0/3000475623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faacf9d60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 -- 172.21.15.103:0/3000475623 >> 172.21.15.103:0/3000475623 conn(0x558faacf2ac0 msgr2=0x558faacf3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe119600640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1040b5360 0x7fe1040b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 -- 172.21.15.103:0/3000475623 shutdown_connections 2024-09-11T00:25:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe119600640 1 -- 172.21.15.103:0/1107077697 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1080e6190 con 0x7fe11c076f90 2024-09-11T00:25:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe12663b740 1 -- 172.21.15.103:0/3000475623 wait complete. 2024-09-11T00:25:02.218 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.206+0000 7fe120c00640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1040b5360 0x7fe1040b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 -- start start 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 Processor -- start 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe120c00640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1040b5360 0x7fe1040b7820 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe110009230 tx=0x7fe1100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faad01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfa2a0 0x558faad017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfcce0 0x558faad023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558faab91ab0 con 0x558faacfcce0 2024-09-11T00:25:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558faac00570 con 0x558faacf7970 2024-09-11T00:25:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558faacffcc0 con 0x558faacfa2a0 2024-09-11T00:25:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.207+0000 7fe12663b740 10 client.0 _ll_register_callbacks cb 0x558faac0f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faad01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faad01130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40338/0 (socket says 172.21.15.103:40338) 2024-09-11T00:25:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 -- 172.21.15.103:0/2790021133 learned_addr learned my addr 172.21.15.103:0/2790021133 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe121600640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfcce0 0x558faad023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe11be00640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfa2a0 0x558faad017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 -- 172.21.15.103:0/2790021133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfa2a0 msgr2=0x558faad017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfa2a0 0x558faad017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 -- 172.21.15.103:0/2790021133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfcce0 msgr2=0x558faad023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfcce0 0x558faad023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 -- 172.21.15.103:0/2790021133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558faa9bb790 con 0x558faacf7970 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe121600640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfcce0 0x558faad023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe120c00640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faad01130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe110009a20 tx=0x7fe110009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a366201e7e0bdb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.208+0000 7fe102000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:25:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe102000640 1 -- 172.21.15.103:0/2790021133 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110035040 con 0x558faacf7970 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe102000640 1 -- 172.21.15.103:0/2790021133 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe110003cf0 con 0x558faacf7970 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe114004050 con 0x558faacf7970 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe102000640 1 -- 172.21.15.103:0/2790021133 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe110051470 con 0x558faacf7970 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe12663b740 10 client.25523 fetch_fsmap learned FSMap version 28 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558faaa34620 con 0x558faacf7970 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe102000640 1 -- 172.21.15.103:0/2790021133 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe114004050 con 0x558faacf7970 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.209+0000 7fe102000640 1 -- 172.21.15.103:0/2790021133 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe110044030 con 0x558faacf7970 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe12663b740 10 client.25523 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe12663b740 10 client.25523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe12663b740 20 client.25523 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe12663b740 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad0c970 0x558faad0ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe12663b740 4 client.25523 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558faac6c600 con 0x558faad0c970 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe11be00640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad0c970 0x558faad0ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe11be00640 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad0c970 0x558faad0ede0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34329ab8fbce1d51 server_cookie=bf87f2f7eea3f557 in_seq=0 out_seq=0 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.210+0000 7fe102000640 10 client.25523 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.211+0000 7fe102000640 1 -- 172.21.15.103:0/2790021133 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558faac6c600 con 0x558faad0c970 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.211+0000 7fe102000640 10 client.25523 handle_command_reply: tid=0 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.211+0000 7fe12663b740 1 client.25523 shutdown 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 msgr2=0x558faad01130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faad01130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe110009a20 tx=0x7fe110009260 comp rx=0 tx=0).stop 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 shutdown_connections 2024-09-11T00:25:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfcce0 0x558faad023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad0c970 0x558faad0ede0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacfa2a0 0x558faad017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 --2- 172.21.15.103:0/2790021133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacf7970 0x558faad01130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 >> 172.21.15.103:0/2790021133 conn(0x558faacf2ac0 msgr2=0x558faacf1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe102000640 0 client.25523 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 shutdown_connections 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 1 -- 172.21.15.103:0/2790021133 wait complete. 2024-09-11T00:25:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.212+0000 7fe12663b740 20 client.25523 trim_cache size 0 max 16384 2024-09-11T00:25:02.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.230+0000 7fe12663b740 1 Processor -- start 2024-09-11T00:25:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.230+0000 7fe12663b740 1 -- start start 2024-09-11T00:25:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe12663b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe12663b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacf3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe12663b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faacf3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe12663b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1aff30 con 0x558faacfa2a0 2024-09-11T00:25:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe12663b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1ad800 con 0x558faacfcce0 2024-09-11T00:25:02.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe12663b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1ab650 con 0x558faacf7970 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe11be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacf3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:46734/0 (socket says 172.21.15.103:46734) 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 -- 172.21.15.103:0/3907900793 learned_addr learned my addr 172.21.15.103:0/3907900793 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe121600640 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faacf3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 -- 172.21.15.103:0/3907900793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 msgr2=0x558faacf3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faacf3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 -- 172.21.15.103:0/3907900793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 msgr2=0x558faacf3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacf3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe120c00640 1 -- 172.21.15.103:0/3907900793 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558faacfa0d0 con 0x558faacf7970 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.231+0000 7fe11be00640 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.232+0000 7fe120c00640 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad002d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110032f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d3150bdeccb3a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.232+0000 7fe10d200640 1 -- 172.21.15.103:0/3907900793 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110063070 con 0x558faacf7970 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.232+0000 7fe10d200640 1 -- 172.21.15.103:0/3907900793 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11004c070 con 0x558faacf7970 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.232+0000 7fe10d200640 1 -- 172.21.15.103:0/3907900793 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110035040 con 0x558faacf7970 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 -- 172.21.15.103:0/3907900793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 msgr2=0x558faad002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad002d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110032f50 comp rx=0 tx=0).stop 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 -- 172.21.15.103:0/3907900793 shutdown_connections 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faacf3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faacf3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 --2- 172.21.15.103:0/3907900793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad002d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 -- 172.21.15.103:0/3907900793 >> 172.21.15.103:0/3907900793 conn(0x558faacf2ac0 msgr2=0x558faacfd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 -- 172.21.15.103:0/3907900793 shutdown_connections 2024-09-11T00:25:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 -- 172.21.15.103:0/3907900793 wait complete. 2024-09-11T00:25:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.233+0000 7fe12663b740 1 -- start start 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 1 Processor -- start 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad07fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faad08520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faad06b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1abcc0 con 0x558faacfa2a0 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1ad130 con 0x558faacfcce0 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c1ac390 con 0x558faacf7970 2024-09-11T00:25:02.240 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe12663b740 10 client.0 _ll_register_callbacks cb 0x7fe110009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe11be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faad08520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe121600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faad06b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe11be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faad08520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:49276/0 (socket says 172.21.15.103:49276) 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe11be00640 1 -- 172.21.15.103:0/3675001005 learned_addr learned my addr 172.21.15.103:0/3675001005 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe121600640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faad06b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40354/0 (socket says 172.21.15.103:40354) 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.234+0000 7fe120c00640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad07fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe11be00640 1 -- 172.21.15.103:0/3675001005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 msgr2=0x558faad07fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe11be00640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad07fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe11be00640 1 -- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 msgr2=0x558faad06b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe11be00640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faad06b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe11be00640 1 -- 172.21.15.103:0/3675001005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558faad07090 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe120c00640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe11be00640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faad08520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1080ac4f0 tx=0x7fe108009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a9cff7408380e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe102000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe102000640 1 -- 172.21.15.103:0/3675001005 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080d3020 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe102000640 1 -- 172.21.15.103:0/3675001005 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe1080a87d0 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe11004c070 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe102000640 1 -- 172.21.15.103:0/3675001005 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe1080acc60 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe12663b740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.235+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558faad7a080 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe102000640 1 -- 172.21.15.103:0/3675001005 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11004c070 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe102000640 1 -- 172.21.15.103:0/3675001005 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 28) ==== 8358+0+0 (secure 0 0 0) 0x7fe1080c71e0 con 0x558faacfa2a0 2024-09-11T00:25:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe12663b740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe12663b740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe12663b740 20 client.15510 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe12663b740 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad7aa10 0x558faad7ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe12663b740 4 client.15510 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558faac680b0 con 0x558faad7aa10 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.236+0000 7fe120c00640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad7aa10 0x558faad7ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.237+0000 7fe120c00640 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad7aa10 0x558faad7ce30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b6082a2acc03e63 server_cookie=745b14d73e000c64 in_seq=0 out_seq=0 2024-09-11T00:25:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.237+0000 7fe102000640 10 client.15510 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe102000640 1 -- 172.21.15.103:0/3675001005 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558faac680b0 con 0x558faad7aa10 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe102000640 10 client.15510 handle_command_reply: tid=0 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 client.15510 shutdown 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 msgr2=0x558faad08520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faad08520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1080ac4f0 tx=0x7fe108009f20 comp rx=0 tx=0).stop 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 shutdown_connections 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558faacfcce0 0x558faad06b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558faacfa2a0 0x558faad08520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558faad7aa10 0x558faad7ce30 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 --2- 172.21.15.103:0/3675001005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558faacf7970 0x558faad07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 >> 172.21.15.103:0/3675001005 conn(0x558faacf2ac0 msgr2=0x558faacf6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe102000640 0 client.15510 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 shutdown_connections 2024-09-11T00:25:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 1 -- 172.21.15.103:0/3675001005 wait complete. 2024-09-11T00:25:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.735+0000 7fe12663b740 20 client.15510 trim_cache size 0 max 16384 2024-09-11T00:25:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:25:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:25:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:25:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:25:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1040b5360 msgr2=0x7fe1040b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1040b5360 0x7fe1040b7820 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe110009230 tx=0x7fe1100058d0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 msgr2=0x7fe11c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 0x7fe11c19b3e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe10800e9f0 tx=0x7fe10800edf0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 shutdown_connections 2024-09-11T00:25:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1040b5360 0x7fe1040b7820 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe11c10a6b0 0x7fe11c19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11c076f90 0x7fe11c19b3e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 --2- 172.21.15.103:0/1107077697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe11c075b20 0x7fe11c19ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 >> 172.21.15.103:0/1107077697 conn(0x7fe11c0fe030 msgr2=0x7fe11c106690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.736+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 shutdown_connections 2024-09-11T00:25:02.741 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:02.737+0000 7fe122000640 1 -- 172.21.15.103:0/1107077697 wait complete. 2024-09-11T00:25:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:02 smithi183 ceph-mon[35934]: pgmap v457: 129 pgs: 129 active+clean; 20 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 964 KiB/s wr, 128 op/s 2024-09-11T00:25:02.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:02 smithi103 ceph-mon[30578]: pgmap v457: 129 pgs: 129 active+clean; 20 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 964 KiB/s wr, 128 op/s 2024-09-11T00:25:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:02 smithi117 ceph-mon[35525]: pgmap v457: 129 pgs: 129 active+clean; 20 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 964 KiB/s wr, 128 op/s 2024-09-11T00:25:02.969 INFO:tasks.workunit.client.0.smithi103.stderr:Updating files: 7% (322/4351) Updating files: 8% (349/4351) Updating files: 9% (392/4351) Updating files: 10% (436/4351) Updating files: 11% (479/4351) Updating files: 12% (523/4351) Updating files: 13% (566/4351) Updating files: 14% (610/4351) Updating files: 15% (653/4351) Updating files: 16% (697/4351) Updating files: 17% (740/4351) Updating files: 18% (784/4351) Updating files: 19% (827/4351) Updating files: 20% (871/4351) Updating files: 21% (914/4351) Updating files: 22% (958/4351) Updating files: 23% (1001/4351) Updating files: 24% (1045/4351) Updating files: 25% (1088/4351) Updating files: 26% (1132/4351) Updating files: 27% (1175/4351) Updating files: 28% (1219/4351) Updating files: 28% (1235/4351) Updating files: 29% (1262/4351) Updating files: 29% (1303/4351) Updating files: 30% (1306/4351) Updating files: 31% (1349/4351) Updating files: 32% (1393/4351) Updating files: 33% (1436/4351) Updating files: 34% (1480/4351) Updating files: 35% (1523/4351) Updating files: 36% (1567/4351) Updating files: 37% (1610/4351) Updating files: 38% (1654/4351) Updating files: 39% (1697/4351) Updating files: 40% (1741/4351) Updating files: 41% (1784/4351) Updating files: 42% (1828/4351) Updating files: 43% (1871/4351) Updating files: 43% (1887/4351) Updating files: 44% (1915/4351) Updating files: 45% (1958/4351) Updating files: 46% (2002/4351) Updating files: 47% (2045/4351) Updating files: 48% (2089/4351) Updating files: 49% (2132/4351) Updating files: 50% (2176/4351) Updating files: 51% (2220/4351) Updating files: 52% (2263/4351) Updating files: 53% (2307/4351) Updating files: 54% (2350/4351) Updating files: 55% (2394/4351) Updating files: 56% (2437/4351) Updating files: 57% (2481/4351) Updating files: 57% (2487/4351) Updating files: 58% (2524/4351) Updating files: 59% (2568/4351) Updating files: 60% (2611/4351) Updating files: 61% (2655/4351) Updating files: 62% (2698/4351) Updating files: 63% (2742/4351) Updating files: 64% (2785/4351) Updating files: 64% (2809/4351) Updating files: 65% (2829/4351) Updating files: 66% (2872/4351) Updating files: 67% (2916/4351) Updating files: 68% (2959/4351) Updating files: 69% (3003/4351) Updating files: 70% (3046/4351) Updating files: 71% (3090/4351) Updating files: 72% (3133/4351) Updating files: 72% (3168/4351) Updating files: 73% (3177/4351) Updating files: 74% (3220/4351) Updating files: 75% (3264/4351) Updating files: 76% (3307/4351) Updating files: 77% (3351/4351) Updating files: 78% (3394/4351) Updating files: 79% (3438/4351) Updating files: 80% (3481/4351) Updating files: 81% (3525/4351) Updating files: 82% (3568/4351) Updating files: 83% (3612/4351) Updating files: 84% (3655/4351) Updating files: 85% (3699/4351) Updating files: 86% (3742/4351) Updating files: 86% (3749/4351) Updating files: 87% (3786/4351) Updating files: 88% (3829/4351) Updating files: 89% (3873/4351) Updating files: 90% (3916/4351) Updating files: 91% (3960/4351) Updating files: 92% (4003/4351) Updating files: 93% (4047/4351) Updating files: 94% (4090/4351) Updating files: 94% (4127/4351) Updating files: 95% (4134/4351) Updating files: 96% (4177/4351) Updating files: 97% (4221/4351) Updating files: 98% (4264/4351) Updating files: 99% (4308/4351) Updating files: 100% (4351/4351) Updating files: 100% (4351/4351), done. 2024-09-11T00:25:02.981 INFO:tasks.workunit.client.0.smithi103.stderr:+ cd xfstests-dev 2024-09-11T00:25:02.981 INFO:tasks.workunit.client.0.smithi103.stderr:+ git checkout 0e5c12dfd008efc2848c98108c9237487e91ef35 2024-09-11T00:25:05.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:05 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:05.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:05 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:05.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:05 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:06.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:06 smithi103 ceph-mon[30578]: pgmap v458: 129 pgs: 129 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.1 MiB/s wr, 280 op/s 2024-09-11T00:25:06.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:06 smithi117 ceph-mon[35525]: pgmap v458: 129 pgs: 129 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.1 MiB/s wr, 280 op/s 2024-09-11T00:25:06.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:06 smithi183 ceph-mon[35934]: pgmap v458: 129 pgs: 129 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.1 MiB/s wr, 280 op/s 2024-09-11T00:25:08.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:08 smithi103 ceph-mon[30578]: pgmap v459: 129 pgs: 129 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 972 KiB/s wr, 313 op/s 2024-09-11T00:25:08.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:08 smithi117 ceph-mon[35525]: pgmap v459: 129 pgs: 129 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 972 KiB/s wr, 313 op/s 2024-09-11T00:25:08.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:08 smithi183 ceph-mon[35934]: pgmap v459: 129 pgs: 129 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 972 KiB/s wr, 313 op/s 2024-09-11T00:25:10.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:10 smithi103 ceph-mon[30578]: pgmap v460: 129 pgs: 129 active+clean; 21 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 992 KiB/s wr, 386 op/s 2024-09-11T00:25:10.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:10 smithi117 ceph-mon[35525]: pgmap v460: 129 pgs: 129 active+clean; 21 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 992 KiB/s wr, 386 op/s 2024-09-11T00:25:10.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:10 smithi183 ceph-mon[35934]: pgmap v460: 129 pgs: 129 active+clean; 21 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 992 KiB/s wr, 386 op/s 2024-09-11T00:25:12.002 INFO:tasks.workunit.client.0.smithi103.stderr:Note: switching to '0e5c12dfd008efc2848c98108c9237487e91ef35'. 2024-09-11T00:25:12.002 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:25:12.002 INFO:tasks.workunit.client.0.smithi103.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: git switch -c 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr:Or undo this operation with: 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: git switch - 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T00:25:12.003 INFO:tasks.workunit.client.0.smithi103.stderr: 2024-09-11T00:25:12.004 INFO:tasks.workunit.client.0.smithi103.stderr:HEAD is now at 0e5c12df xfs/522: use reflink instead of crc as test feature 2024-09-11T00:25:12.011 INFO:tasks.workunit.client.0.smithi103.stderr:+ make -j4 2024-09-11T00:25:12.027 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize -cfi 2024-09-11T00:25:12.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:12 smithi103 ceph-mon[30578]: pgmap v461: 129 pgs: 129 active+clean; 31 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 396 op/s 2024-09-11T00:25:12.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:12 smithi117 ceph-mon[35525]: pgmap v461: 129 pgs: 129 active+clean; 31 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 396 op/s 2024-09-11T00:25:12.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:12 smithi183 ceph-mon[35934]: pgmap v461: 129 pgs: 129 active+clean; 31 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 396 op/s 2024-09-11T00:25:12.615 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: putting auxiliary files in '.'. 2024-09-11T00:25:12.615 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file './config.guess' 2024-09-11T00:25:12.626 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file './config.sub' 2024-09-11T00:25:12.638 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file './install-sh' 2024-09-11T00:25:12.650 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file './ltmain.sh' 2024-09-11T00:25:12.711 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-09-11T00:25:12.712 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-09-11T00:25:12.725 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-09-11T00:25:12.738 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-09-11T00:25:12.753 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-09-11T00:25:12.764 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-09-11T00:25:12.780 INFO:tasks.workunit.client.0.smithi103.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-09-11T00:25:12.783 INFO:tasks.workunit.client.0.smithi103.stdout:cp include/install-sh . 2024-09-11T00:25:12.795 INFO:tasks.workunit.client.0.smithi103.stdout:aclocal -I m4 2024-09-11T00:25:14.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:14 smithi103 ceph-mon[30578]: pgmap v462: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-09-11T00:25:14.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:14 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:14.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:14 smithi117 ceph-mon[35525]: pgmap v462: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-09-11T00:25:14.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:14 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:14.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:14 smithi183 ceph-mon[35934]: pgmap v462: 129 pgs: 129 active+clean; 33 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 430 op/s 2024-09-11T00:25:14.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:14 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 -- 172.21.15.103:0/1310562384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101027b0 msgr2=0x7f0c10102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 --2- 172.21.15.103:0/1310562384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101027b0 0x7f0c10102bb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009ca0 tx=0x7f0c0002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 -- 172.21.15.103:0/1310562384 shutdown_connections 2024-09-11T00:25:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 --2- 172.21.15.103:0/1310562384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c101042c0 0x7f0c1010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 --2- 172.21.15.103:0/1310562384 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 0x7f0c10103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 --2- 172.21.15.103:0/1310562384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101027b0 0x7f0c10102bb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 -- 172.21.15.103:0/1310562384 >> 172.21.15.103:0/1310562384 conn(0x7f0c100fe050 msgr2=0x7f0c10100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.889+0000 7f0c15400640 1 -- 172.21.15.103:0/1310562384 shutdown_connections 2024-09-11T00:25:15.893 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.890+0000 7f0c15400640 1 -- 172.21.15.103:0/1310562384 wait complete. 2024-09-11T00:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.890+0000 7f0c15400640 1 Processor -- start 2024-09-11T00:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.890+0000 7f0c15400640 1 -- start start 2024-09-11T00:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c15400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c101027b0 0x7f0c1019adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c15400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 0x7f0c1019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c15400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101042c0 0x7f0c1019c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c15400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010e470 con 0x7f0c101027b0 2024-09-11T00:25:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c15400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010e2f0 con 0x7f0c101042c0 2024-09-11T00:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c15400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010e5f0 con 0x7f0c10103900 2024-09-11T00:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 0x7f0c1019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c101027b0 0x7f0c1019adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 0x7f0c1019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:49244/0 (socket says 172.21.15.103:49244) 2024-09-11T00:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 -- 172.21.15.103:0/4096969766 learned_addr learned my addr 172.21.15.103:0/4096969766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c14a00640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101042c0 0x7f0c1019c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.896 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 -- 172.21.15.103:0/4096969766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101042c0 msgr2=0x7f0c1019c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101042c0 0x7f0c1019c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 -- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c101027b0 msgr2=0x7f0c1019adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c101027b0 0x7f0c1019adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c0f400640 1 -- 172.21.15.103:0/4096969766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1019c900 con 0x7f0c10103900 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.891+0000 7f0c14a00640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101042c0 0x7f0c1019c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.892+0000 7f0c0f400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 0x7f0c1019b410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e9f0 tx=0x7f0bf800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.892+0000 7f0c0cc00640 1 -- 172.21.15.103:0/4096969766 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800cd80 con 0x7f0c10103900 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.892+0000 7f0c0cc00640 1 -- 172.21.15.103:0/4096969766 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf80054d0 con 0x7f0c10103900 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.892+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1019cbf0 con 0x7f0c10103900 2024-09-11T00:25:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.892+0000 7f0c0cc00640 1 -- 172.21.15.103:0/4096969766 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8010690 con 0x7f0c10103900 2024-09-11T00:25:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.892+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a32c0 con 0x7f0c10103900 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 Processor -- start 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 -- start start 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534bf9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bfc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558534b676b0 con 0x558534bf4970 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558534b56ae0 con 0x558534bf9ce0 2024-09-11T00:25:15.905 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c19866740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558534b5a8f0 con 0x558534bf72a0 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.895+0000 7f0c0cc00640 1 -- 172.21.15.103:0/4096969766 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0bf8004e90 con 0x7f0c10103900 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534bf9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534bf9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:49252/0 (socket says 172.21.15.103:49252) 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 -- 172.21.15.103:0/2341595910 learned_addr learned my addr 172.21.15.103:0/2341595910 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c14a00640 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0f400640 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bfc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 -- 172.21.15.103:0/2341595910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 msgr2=0x558534bfc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bfc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 -- 172.21.15.103:0/2341595910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 msgr2=0x558534bf6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 -- 172.21.15.103:0/2341595910 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558534b23780 con 0x558534bf72a0 2024-09-11T00:25:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.896+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534bf9690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c00031a10 tx=0x7f0c00005f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36d96d0a9da6a018 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c05200640 1 -- 172.21.15.103:0/2341595910 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00031de0 con 0x558534bf72a0 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c05200640 1 -- 172.21.15.103:0/2341595910 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c00036030 con 0x558534bf72a0 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c05200640 1 -- 172.21.15.103:0/2341595910 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00004240 con 0x558534bf72a0 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c19866740 1 -- 172.21.15.103:0/2341595910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 msgr2=0x558534bf9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c19866740 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534bf9690 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0c00031a10 tx=0x7f0c00005f60 comp rx=0 tx=0).stop 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c0cc00640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bf40b5360 0x7f0bf40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c19866740 1 -- 172.21.15.103:0/2341595910 shutdown_connections 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c19866740 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bfc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c19866740 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534bf9690 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c19866740 1 --2- 172.21.15.103:0/2341595910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c19866740 1 -- 172.21.15.103:0/2341595910 >> 172.21.15.103:0/2341595910 conn(0x558534befac0 msgr2=0x558534bf0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.897+0000 7f0c0cc00640 1 -- 172.21.15.103:0/4096969766 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0bf80e63d0 con 0x7f0c10103900 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.898+0000 7f0c19866740 1 -- 172.21.15.103:0/2341595910 shutdown_connections 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.898+0000 7f0c14a00640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bf40b5360 0x7f0bf40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.898+0000 7f0c19866740 1 -- 172.21.15.103:0/2341595910 wait complete. 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.898+0000 7f0c19866740 1 -- start start 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.898+0000 7f0c19866740 1 Processor -- start 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.898+0000 7f0c14a00640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bf40b5360 0x7f0bf40b7820 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c1019bb40 tx=0x7f0c08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c19866740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf4970 0x558534bfde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c19866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf72a0 0x558534bfe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c19866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c19866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558534a8eab0 con 0x558534bf72a0 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c19866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558534afd570 con 0x558534bf9ce0 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c19866740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558534bfcae0 con 0x558534bf4970 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c19866740 10 client.0 _ll_register_callbacks cb 0x558534b0c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf4970 0x558534bfde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c14a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf72a0 0x558534bfe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf4970 0x558534bfde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:49254/0 (socket says 172.21.15.103:49254) 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 -- 172.21.15.103:0/87617245 learned_addr learned my addr 172.21.15.103:0/87617245 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 -- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 msgr2=0x558534bff0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bff0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 -- 172.21.15.103:0/87617245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf72a0 msgr2=0x558534bfe4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf72a0 0x558534bfe4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0fe00640 1 -- 172.21.15.103:0/87617245 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585348b8790 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c14a00640 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.899+0000 7f0c0f400640 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf72a0 0x558534bfe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf4970 0x558534bfde30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c00000f80 tx=0x7f0c00003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d007bc4a17afbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0bfd600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0bfd600640 1 -- 172.21.15.103:0/87617245 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00052030 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c00036030 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0bfd600640 1 -- 172.21.15.103:0/87617245 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0c00031450 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0bfd600640 1 -- 172.21.15.103:0/87617245 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c0005c050 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0c19866740 10 client.25567 fetch_fsmap learned FSMap version 30 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558534931620 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.900+0000 7f0bfd600640 1 -- 172.21.15.103:0/87617245 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c00036030 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0bfd600640 1 -- 172.21.15.103:0/87617245 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0c0004d030 con 0x558534bf4970 2024-09-11T00:25:15.910 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c19866740 10 client.25567 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c19866740 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c19866740 20 client.25567 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c19866740 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c09490 0x558534c0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c19866740 4 client.25567 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558534b69600 con 0x558534c09490 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c0f400640 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c09490 0x558534c0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0bfd600640 10 client.25567 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.901+0000 7f0c0f400640 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c09490 0x558534c0b880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c472d6c68d1d02e0 server_cookie=12f86bc51af54b0e in_seq=0 out_seq=0 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.902+0000 7f0bfd600640 1 -- 172.21.15.103:0/87617245 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558534b69600 con 0x558534c09490 2024-09-11T00:25:15.911 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.902+0000 7f0bfd600640 10 client.25567 handle_command_reply: tid=0 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.902+0000 7f0c19866740 1 client.25567 shutdown 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.902+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf4970 msgr2=0x558534bfde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.902+0000 7f0c19866740 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf4970 0x558534bfde30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0c00000f80 tx=0x7f0c00003c20 comp rx=0 tx=0).stop 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.902+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 shutdown_connections 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.902+0000 7f0c19866740 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534bff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0c19866740 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf72a0 0x558534bfe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0c19866740 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c09490 0x558534c0b880 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0c19866740 1 --2- 172.21.15.103:0/87617245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf4970 0x558534bfde30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0bfd600640 0 client.25567 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 >> 172.21.15.103:0/87617245 conn(0x558534befac0 msgr2=0x558534beebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 shutdown_connections 2024-09-11T00:25:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0c19866740 1 -- 172.21.15.103:0/87617245 wait complete. 2024-09-11T00:25:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.903+0000 7f0c19866740 20 client.25567 trim_cache size 0 max 16384 2024-09-11T00:25:15.921 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 Processor -- start 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 -- start start 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf72a0 0x558534bf42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf9ce0 0x558534bf18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101afea0 con 0x558534bf4970 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101ad740 con 0x558534bf72a0 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.918+0000 7f0c19866740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101ab590 con 0x558534bf9ce0 2024-09-11T00:25:15.922 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf72a0 0x558534bf42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c14a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf9ce0 0x558534bf18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf72a0 0x558534bf42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:43092/0 (socket says 172.21.15.103:43092) 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 -- 172.21.15.103:0/2945728601 learned_addr learned my addr 172.21.15.103:0/2945728601 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 -- 172.21.15.103:0/2945728601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf9ce0 msgr2=0x558534bf18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf9ce0 0x558534bf18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 -- 172.21.15.103:0/2945728601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 msgr2=0x558534bf3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 -- 172.21.15.103:0/2945728601 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558534c0bbf0 con 0x558534bf72a0 2024-09-11T00:25:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0c0f400640 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf72a0 0x558534bf42e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80e3ad0 tx=0x7f0bf80e3a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa69be19fedc1483 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.919+0000 7f0bffe00640 1 -- 172.21.15.103:0/2945728601 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80a86c0 con 0x558534bf72a0 2024-09-11T00:25:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0bffe00640 1 -- 172.21.15.103:0/2945728601 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf80ac330 con 0x558534bf72a0 2024-09-11T00:25:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0bffe00640 1 -- 172.21.15.103:0/2945728601 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80abf90 con 0x558534bf72a0 2024-09-11T00:25:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 -- 172.21.15.103:0/2945728601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf72a0 msgr2=0x558534bf42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf72a0 0x558534bf42e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80e3ad0 tx=0x7f0bf80e3a30 comp rx=0 tx=0).stop 2024-09-11T00:25:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 -- 172.21.15.103:0/2945728601 shutdown_connections 2024-09-11T00:25:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf9ce0 0x558534bf18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.927 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf72a0 0x558534bf42e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 --2- 172.21.15.103:0/2945728601 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534bf3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 -- 172.21.15.103:0/2945728601 >> 172.21.15.103:0/2945728601 conn(0x558534befac0 msgr2=0x558534bfa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 -- 172.21.15.103:0/2945728601 shutdown_connections 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.920+0000 7f0c19866740 1 -- 172.21.15.103:0/2945728601 wait complete. 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 -- start start 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 Processor -- start 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534c05ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534c047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534c04ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101abc00 con 0x558534bf4970 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101ad070 con 0x558534bf9ce0 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0c101ac2d0 con 0x558534bf72a0 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c19866740 10 client.0 _ll_register_callbacks cb 0x7f0c080095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.921+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534c05ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c14a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534c04ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c0fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534c05ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39322/0 (socket says 172.21.15.103:39322) 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c0f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534c047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c0fe00640 1 -- 172.21.15.103:0/2462130451 learned_addr learned my addr 172.21.15.103:0/2462130451 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c14a00640 1 -- 172.21.15.103:0/2462130451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 msgr2=0x558534c047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c14a00640 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534c047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c14a00640 1 -- 172.21.15.103:0/2462130451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 msgr2=0x558534c05ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c14a00640 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534c05ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c14a00640 1 -- 172.21.15.103:0/2462130451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558534c032f0 con 0x558534bf9ce0 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534c05ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c14a00640 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534c04ec0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x558534bfeb80 tx=0x7f0c080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab1b561a731fe8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0bfd600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0bfd600640 1 -- 172.21.15.103:0/2462130451 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08020e50 con 0x558534bf9ce0 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0bfd600640 1 -- 172.21.15.103:0/2462130451 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0c08018030 con 0x558534bf9ce0 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.922+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf80ac330 con 0x558534bf9ce0 2024-09-11T00:25:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0bfd600640 1 -- 172.21.15.103:0/2462130451 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c08039050 con 0x558534bf9ce0 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 10 client.25547 fetch_fsmap learned FSMap version 30 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558534c76650 con 0x558534bf9ce0 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0bfd600640 1 -- 172.21.15.103:0/2462130451 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bf80ac330 con 0x558534bf9ce0 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0bfd600640 1 -- 172.21.15.103:0/2462130451 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0c0802c410 con 0x558534bf9ce0 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 10 client.25547 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 20 client.25547 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c77110 0x558534c79530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 4 client.25547 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558534b650b0 con 0x558534c77110 2024-09-11T00:25:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.923+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c77110 0x558534c79530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.924+0000 7f0bfd600640 10 client.25547 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:15.924+0000 7f0c0fe00640 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c77110 0x558534c79530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ebf27ca94c2160 server_cookie=65801602e24b5494 in_seq=0 out_seq=0 2024-09-11T00:25:16.421 INFO:tasks.workunit.client.0.smithi103.stdout:autoheader 2024-09-11T00:25:16.422 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:16 smithi103 ceph-mon[30578]: pgmap v463: 129 pgs: 129 active+clean; 39 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 555 op/s 2024-09-11T00:25:16.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:16 smithi117 ceph-mon[35525]: pgmap v463: 129 pgs: 129 active+clean; 39 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 555 op/s 2024-09-11T00:25:16.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:16 smithi183 ceph-mon[35934]: pgmap v463: 129 pgs: 129 active+clean; 39 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 555 op/s 2024-09-11T00:25:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.854+0000 7f0bfd600640 1 -- 172.21.15.103:0/2462130451 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558534b650b0 con 0x558534c77110 2024-09-11T00:25:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.854+0000 7f0bfd600640 10 client.25547 handle_command_reply: tid=0 2024-09-11T00:25:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.854+0000 7f0c19866740 1 client.25547 shutdown 2024-09-11T00:25:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.857+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 msgr2=0x558534c04ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.857+0000 7f0c19866740 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534c04ec0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x558534bfeb80 tx=0x7f0c080046c0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 shutdown_connections 2024-09-11T00:25:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558534bf9ce0 0x558534c04ec0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558534c77110 0x558534c79530 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558534bf72a0 0x558534c047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 --2- 172.21.15.103:0/2462130451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558534bf4970 0x558534c05ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 >> 172.21.15.103:0/2462130451 conn(0x558534befac0 msgr2=0x558534bfa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0bfd600640 0 client.25547 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 shutdown_connections 2024-09-11T00:25:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.861+0000 7f0c19866740 1 -- 172.21.15.103:0/2462130451 wait complete. 2024-09-11T00:25:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.862+0000 7f0c19866740 20 client.25547 trim_cache size 0 max 16384 2024-09-11T00:25:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:25:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:25:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:25:16.863 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:25:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bf40b5360 msgr2=0x7f0bf40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bf40b5360 0x7f0bf40b7820 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c1019bb40 tx=0x7f0c08005f90 comp rx=0 tx=0).stop 2024-09-11T00:25:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 msgr2=0x7f0c1019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 0x7f0c1019b410 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e9f0 tx=0x7f0bf800edf0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 shutdown_connections 2024-09-11T00:25:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0bf40b5360 0x7f0bf40b7820 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c101042c0 0x7f0c1019c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0c10103900 0x7f0c1019b410 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 --2- 172.21.15.103:0/4096969766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0c101027b0 0x7f0c1019adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 >> 172.21.15.103:0/4096969766 conn(0x7f0c100fe050 msgr2=0x7f0c10109020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 shutdown_connections 2024-09-11T00:25:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:16.863+0000 7f0c15400640 1 -- 172.21.15.103:0/4096969766 wait complete. 2024-09-11T00:25:16.940 INFO:tasks.workunit.client.0.smithi103.stdout:autoconf 2024-09-11T00:25:17.218 INFO:tasks.workunit.client.0.smithi103.stdout:./configure \ 2024-09-11T00:25:17.218 INFO:tasks.workunit.client.0.smithi103.stdout: --libexecdir=/usr/lib \ 2024-09-11T00:25:17.218 INFO:tasks.workunit.client.0.smithi103.stdout: --exec_prefix=/var/lib 2024-09-11T00:25:17.281 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:25:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setuser ceph since I am not root 2024-09-11T00:25:17.281 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:25:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setgroup ceph since I am not root 2024-09-11T00:25:17.281 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:25:16 smithi183 ceph-mds[65455]: -- 172.21.15.183:0/42200570 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562a42cef860 con 0x562a42ccd400 2024-09-11T00:25:17.281 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:25:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: starting mds.f at 2024-09-11T00:25:17.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:17 smithi183 ceph-mon[35934]: daemon mds.f restarted 2024-09-11T00:25:17.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:17 smithi103 ceph-mon[30578]: daemon mds.f restarted 2024-09-11T00:25:17.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:17 smithi117 ceph-mon[35525]: daemon mds.f restarted 2024-09-11T00:25:17.739 INFO:tasks.workunit.client.0.smithi103.stdout:checking build system type... x86_64-pc-linux-gnu 2024-09-11T00:25:17.739 INFO:tasks.workunit.client.0.smithi103.stdout:checking host system type... x86_64-pc-linux-gnu 2024-09-11T00:25:17.745 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to print strings... printf 2024-09-11T00:25:17.746 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc... gcc 2024-09-11T00:25:17.951 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether the C compiler works... yes 2024-09-11T00:25:17.951 INFO:tasks.workunit.client.0.smithi103.stdout:checking for C compiler default output file name... a.out 2024-09-11T00:25:18.037 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of executables... 2024-09-11T00:25:18.101 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are cross compiling... no 2024-09-11T00:25:18.156 INFO:tasks.workunit.client.0.smithi103.stdout:checking for suffix of object files... o 2024-09-11T00:25:18.196 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are using the GNU C compiler... yes 2024-09-11T00:25:18.237 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc accepts -g... yes 2024-09-11T00:25:18.344 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-11T00:25:18.351 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-09-11T00:25:18.358 INFO:tasks.workunit.client.0.smithi103.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-11T00:25:18.361 INFO:tasks.workunit.client.0.smithi103.stdout:checking for egrep... /usr/bin/grep -E 2024-09-11T00:25:18.363 INFO:tasks.workunit.client.0.smithi103.stdout:checking for fgrep... /usr/bin/grep -F 2024-09-11T00:25:18.371 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ld used by gcc... /usr/bin/ld 2024-09-11T00:25:18.375 INFO:tasks.workunit.client.0.smithi103.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-09-11T00:25:18.383 INFO:tasks.workunit.client.0.smithi103.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-09-11T00:25:18.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:18 smithi103 ceph-mon[30578]: pgmap v464: 129 pgs: 129 active+clean; 44 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 430 op/s 2024-09-11T00:25:18.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:18 smithi103 ceph-mon[30578]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T00:25:18.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:18 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6834/3202362307,v1:172.21.15.183:6835/3202362307] up:boot 2024-09-11T00:25:18.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:18 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T00:25:18.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:18 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:25:18.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:18 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:18.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:18 smithi117 ceph-mon[35525]: pgmap v464: 129 pgs: 129 active+clean; 44 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 430 op/s 2024-09-11T00:25:18.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:18 smithi117 ceph-mon[35525]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T00:25:18.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:18 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6834/3202362307,v1:172.21.15.183:6835/3202362307] up:boot 2024-09-11T00:25:18.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:18 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T00:25:18.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:18 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:25:18.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:18 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:18.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:18 smithi183 ceph-mon[35934]: pgmap v464: 129 pgs: 129 active+clean; 44 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 430 op/s 2024-09-11T00:25:18.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:18 smithi183 ceph-mon[35934]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T00:25:18.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:18 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6834/3202362307,v1:172.21.15.183:6835/3202362307] up:boot 2024-09-11T00:25:18.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:18 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T00:25:18.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:18 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:25:18.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:18 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:19.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:19 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:19.497 INFO:tasks.workunit.client.0.smithi103.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-09-11T00:25:19.497 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether ln -s works... yes 2024-09-11T00:25:19.509 INFO:tasks.workunit.client.0.smithi103.stdout:checking the maximum length of command line arguments... 1572864 2024-09-11T00:25:19.516 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-09-11T00:25:19.516 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-09-11T00:25:19.516 INFO:tasks.workunit.client.0.smithi103.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-09-11T00:25:19.516 INFO:tasks.workunit.client.0.smithi103.stdout:checking for objdump... objdump 2024-09-11T00:25:19.516 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to recognize dependent libraries... pass_all 2024-09-11T00:25:19.517 INFO:tasks.workunit.client.0.smithi103.stdout:checking for dlltool... no 2024-09-11T00:25:19.517 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-09-11T00:25:19.517 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ar... ar 2024-09-11T00:25:19.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:19 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:19.664 INFO:tasks.workunit.client.0.smithi103.stdout:checking for archiver @FILE support... @ 2024-09-11T00:25:19.664 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strip... strip 2024-09-11T00:25:19.664 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ranlib... ranlib 2024-09-11T00:25:19.664 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gawk... gawk 2024-09-11T00:25:19.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:19 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:19.822 INFO:tasks.workunit.client.0.smithi103.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-09-11T00:25:19.825 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sysroot... no 2024-09-11T00:25:19.840 INFO:tasks.workunit.client.0.smithi103.stdout:checking for a working dd... /usr/bin/dd 2024-09-11T00:25:19.855 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-09-11T00:25:19.901 INFO:tasks.workunit.client.0.smithi103.stdout:checking for mt... no 2024-09-11T00:25:19.917 INFO:tasks.workunit.client.0.smithi103.stdout:checking if : is a manifest tool... no 2024-09-11T00:25:20.028 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to run the C preprocessor... gcc -E 2024-09-11T00:25:20.459 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ANSI C header files... yes 2024-09-11T00:25:20.516 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/types.h... yes 2024-09-11T00:25:20.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:20 smithi183 ceph-mon[35934]: pgmap v466: 129 pgs: 129 active+clean; 45 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 712 op/s 2024-09-11T00:25:20.582 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/stat.h... yes 2024-09-11T00:25:20.647 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdlib.h... yes 2024-09-11T00:25:20.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:20 smithi103 ceph-mon[30578]: pgmap v466: 129 pgs: 129 active+clean; 45 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 712 op/s 2024-09-11T00:25:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:20 smithi117 ceph-mon[35525]: pgmap v466: 129 pgs: 129 active+clean; 45 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 712 op/s 2024-09-11T00:25:20.713 INFO:tasks.workunit.client.0.smithi103.stdout:checking for string.h... yes 2024-09-11T00:25:20.780 INFO:tasks.workunit.client.0.smithi103.stdout:checking for memory.h... yes 2024-09-11T00:25:20.846 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strings.h... yes 2024-09-11T00:25:20.913 INFO:tasks.workunit.client.0.smithi103.stdout:checking for inttypes.h... yes 2024-09-11T00:25:20.979 INFO:tasks.workunit.client.0.smithi103.stdout:checking for stdint.h... yes 2024-09-11T00:25:21.046 INFO:tasks.workunit.client.0.smithi103.stdout:checking for unistd.h... yes 2024-09-11T00:25:21.116 INFO:tasks.workunit.client.0.smithi103.stdout:checking for dlfcn.h... yes 2024-09-11T00:25:21.126 INFO:tasks.workunit.client.0.smithi103.stdout:checking for objdir... .libs 2024-09-11T00:25:21.301 INFO:tasks.workunit.client.0.smithi103.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-09-11T00:25:21.301 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-09-11T00:25:21.357 INFO:tasks.workunit.client.0.smithi103.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-09-11T00:25:21.431 INFO:tasks.workunit.client.0.smithi103.stdout:checking if gcc static flag -static works... no 2024-09-11T00:25:21.522 INFO:tasks.workunit.client.0.smithi103.stdout:checking if gcc supports -c -o file.o... yes 2024-09-11T00:25:21.522 INFO:tasks.workunit.client.0.smithi103.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-09-11T00:25:21.536 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-09-11T00:25:21.642 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether -lc should be explicitly linked in... no 2024-09-11T00:25:21.747 INFO:tasks.workunit.client.0.smithi103.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-09-11T00:25:21.747 INFO:tasks.workunit.client.0.smithi103.stdout:checking how to hardcode library paths into programs... immediate 2024-09-11T00:25:21.750 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether stripping libraries is possible... yes 2024-09-11T00:25:21.750 INFO:tasks.workunit.client.0.smithi103.stdout:checking if libtool supports shared libraries... yes 2024-09-11T00:25:21.750 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether to build shared libraries... yes 2024-09-11T00:25:21.750 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether to build static libraries... yes 2024-09-11T00:25:21.764 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc... (cached) gcc 2024-09-11T00:25:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-11T00:25:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-11T00:25:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-11T00:25:21.819 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gmake... /usr/bin/gmake 2024-09-11T00:25:21.820 INFO:tasks.workunit.client.0.smithi103.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-09-11T00:25:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:checking for tar... /bin/tar 2024-09-11T00:25:21.821 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gzip... /bin/gzip 2024-09-11T00:25:21.852 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether gcc -MM is supported... yes 2024-09-11T00:25:21.852 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sort... /bin/sort 2024-09-11T00:25:21.852 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether ln -s works... yes 2024-09-11T00:25:21.852 INFO:tasks.workunit.client.0.smithi103.stdout:checking for rpm... /bin/rpm 2024-09-11T00:25:21.859 INFO:tasks.workunit.client.0.smithi103.stdout:checking for rpmbuild... no 2024-09-11T00:25:21.934 INFO:tasks.workunit.client.0.smithi103.stdout:checking assert.h usability... yes 2024-09-11T00:25:21.960 INFO:tasks.workunit.client.0.smithi103.stdout:checking assert.h presence... yes 2024-09-11T00:25:21.960 INFO:tasks.workunit.client.0.smithi103.stdout:checking for assert.h... yes 2024-09-11T00:25:22.038 INFO:tasks.workunit.client.0.smithi103.stdout:checking bstring.h usability... no 2024-09-11T00:25:22.080 INFO:tasks.workunit.client.0.smithi103.stdout:checking bstring.h presence... no 2024-09-11T00:25:22.080 INFO:tasks.workunit.client.0.smithi103.stdout:checking for bstring.h... no 2024-09-11T00:25:22.149 INFO:tasks.workunit.client.0.smithi103.stdout:checking libgen.h usability... yes 2024-09-11T00:25:22.173 INFO:tasks.workunit.client.0.smithi103.stdout:checking libgen.h presence... yes 2024-09-11T00:25:22.174 INFO:tasks.workunit.client.0.smithi103.stdout:checking for libgen.h... yes 2024-09-11T00:25:22.258 INFO:tasks.workunit.client.0.smithi103.stdout:checking dirent.h usability... yes 2024-09-11T00:25:22.286 INFO:tasks.workunit.client.0.smithi103.stdout:checking dirent.h presence... yes 2024-09-11T00:25:22.287 INFO:tasks.workunit.client.0.smithi103.stdout:checking for dirent.h... yes 2024-09-11T00:25:22.360 INFO:tasks.workunit.client.0.smithi103.stdout:checking errno.h usability... yes 2024-09-11T00:25:22.387 INFO:tasks.workunit.client.0.smithi103.stdout:checking errno.h presence... yes 2024-09-11T00:25:22.387 INFO:tasks.workunit.client.0.smithi103.stdout:checking for errno.h... yes 2024-09-11T00:25:22.460 INFO:tasks.workunit.client.0.smithi103.stdout:checking malloc.h usability... yes 2024-09-11T00:25:22.489 INFO:tasks.workunit.client.0.smithi103.stdout:checking malloc.h presence... yes 2024-09-11T00:25:22.489 INFO:tasks.workunit.client.0.smithi103.stdout:checking for malloc.h... yes 2024-09-11T00:25:22.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:22 smithi183 ceph-mon[35934]: pgmap v467: 129 pgs: 129 active+clean; 46 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 1.29k op/s 2024-09-11T00:25:22.567 INFO:tasks.workunit.client.0.smithi103.stdout:checking uuid.h usability... no 2024-09-11T00:25:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:checking uuid.h presence... no 2024-09-11T00:25:22.609 INFO:tasks.workunit.client.0.smithi103.stdout:checking for uuid.h... no 2024-09-11T00:25:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:22 smithi103 ceph-mon[30578]: pgmap v467: 129 pgs: 129 active+clean; 46 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 1.29k op/s 2024-09-11T00:25:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:22 smithi117 ceph-mon[35525]: pgmap v467: 129 pgs: 129 active+clean; 46 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 1.29k op/s 2024-09-11T00:25:22.688 INFO:tasks.workunit.client.0.smithi103.stdout:checking uuid/uuid.h usability... yes 2024-09-11T00:25:22.716 INFO:tasks.workunit.client.0.smithi103.stdout:checking uuid/uuid.h presence... yes 2024-09-11T00:25:22.717 INFO:tasks.workunit.client.0.smithi103.stdout:checking for uuid/uuid.h... yes 2024-09-11T00:25:22.794 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/uuid.h usability... no 2024-09-11T00:25:22.838 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/uuid.h presence... no 2024-09-11T00:25:22.838 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/uuid.h... no 2024-09-11T00:25:22.909 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/file.h usability... yes 2024-09-11T00:25:22.938 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/file.h presence... yes 2024-09-11T00:25:22.938 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/file.h... yes 2024-09-11T00:25:23.026 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fcntl.h usability... yes 2024-09-11T00:25:23.053 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fcntl.h presence... yes 2024-09-11T00:25:23.054 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/fcntl.h... yes 2024-09-11T00:25:23.135 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/syssgi.h usability... no 2024-09-11T00:25:23.185 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/syssgi.h presence... no 2024-09-11T00:25:23.185 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/syssgi.h... no 2024-09-11T00:25:23.303 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/param.h usability... yes 2024-09-11T00:25:23.338 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/param.h presence... yes 2024-09-11T00:25:23.338 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/param.h... yes 2024-09-11T00:25:23.362 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/stat.h... (cached) yes 2024-09-11T00:25:23.443 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/statvfs.h usability... yes 2024-09-11T00:25:23.469 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/statvfs.h presence... yes 2024-09-11T00:25:23.470 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/statvfs.h... yes 2024-09-11T00:25:23.543 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h usability... yes 2024-09-11T00:25:23.570 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/time.h presence... yes 2024-09-11T00:25:23.570 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/time.h... yes 2024-09-11T00:25:23.644 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/ioctl.h usability... yes 2024-09-11T00:25:23.671 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/ioctl.h presence... yes 2024-09-11T00:25:23.671 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/ioctl.h... yes 2024-09-11T00:25:23.761 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/wait.h usability... yes 2024-09-11T00:25:23.792 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/wait.h presence... yes 2024-09-11T00:25:23.792 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/wait.h... yes 2024-09-11T00:25:23.813 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/types.h... (cached) yes 2024-09-11T00:25:23.825 INFO:tasks.workunit.client.0.smithi103.stdout:checking for strings.h... (cached) yes 2024-09-11T00:25:23.890 INFO:tasks.workunit.client.0.smithi103.stdout:checking err.h usability... yes 2024-09-11T00:25:23.917 INFO:tasks.workunit.client.0.smithi103.stdout:checking err.h presence... yes 2024-09-11T00:25:23.918 INFO:tasks.workunit.client.0.smithi103.stdout:checking for err.h... yes 2024-09-11T00:25:23.991 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/falloc.h usability... yes 2024-09-11T00:25:24.016 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/falloc.h presence... yes 2024-09-11T00:25:24.017 INFO:tasks.workunit.client.0.smithi103.stdout:checking for linux/falloc.h... yes 2024-09-11T00:25:24.097 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-09-11T00:25:24.143 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-09-11T00:25:24.143 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/fs/xfs_fsops.h... no 2024-09-11T00:25:24.211 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fs/xfs_itable.h usability... no 2024-09-11T00:25:24.253 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/fs/xfs_itable.h presence... no 2024-09-11T00:25:24.253 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/fs/xfs_itable.h... no 2024-09-11T00:25:24.296 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:25:24.296 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-11T00:25:24.296 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:25:24.322 INFO:tasks.workunit.client.0.smithi103.stdout:checking xfs/platform_defs.h usability... no 2024-09-11T00:25:24.364 INFO:tasks.workunit.client.0.smithi103.stdout:checking xfs/platform_defs.h presence... no 2024-09-11T00:25:24.365 INFO:tasks.workunit.client.0.smithi103.stdout:checking for xfs/platform_defs.h... no 2024-09-11T00:25:24.432 INFO:tasks.workunit.client.0.smithi103.stdout:checking btrfs/ioctl.h usability... no 2024-09-11T00:25:24.474 INFO:tasks.workunit.client.0.smithi103.stdout:checking btrfs/ioctl.h presence... no 2024-09-11T00:25:24.474 INFO:tasks.workunit.client.0.smithi103.stdout:checking for btrfs/ioctl.h... no 2024-09-11T00:25:24.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:24 smithi183 ceph-mon[35934]: pgmap v468: 129 pgs: 129 active+clean; 45 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-09-11T00:25:24.541 INFO:tasks.workunit.client.0.smithi103.stdout:checking cifs/ioctl.h usability... no 2024-09-11T00:25:24.586 INFO:tasks.workunit.client.0.smithi103.stdout:checking cifs/ioctl.h presence... no 2024-09-11T00:25:24.586 INFO:tasks.workunit.client.0.smithi103.stdout:checking for cifs/ioctl.h... no 2024-09-11T00:25:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:24 smithi103 ceph-mon[30578]: pgmap v468: 129 pgs: 129 active+clean; 45 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-09-11T00:25:24.670 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/mman.h usability... yes 2024-09-11T00:25:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:24 smithi117 ceph-mon[35525]: pgmap v468: 129 pgs: 129 active+clean; 45 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 1.38k op/s 2024-09-11T00:25:24.688 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:24.697 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/mman.h presence... yes 2024-09-11T00:25:24.698 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/mman.h... yes 2024-09-11T00:25:24.794 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/ext4.h usability... no 2024-09-11T00:25:24.837 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/ext4.h presence... no 2024-09-11T00:25:24.837 INFO:tasks.workunit.client.0.smithi103.stdout:checking for linux/ext4.h... no 2024-09-11T00:25:24.888 INFO:tasks.workunit.client.0.smithi103.stdout:checking for xfs/xfs_log_format.h... no 2024-09-11T00:25:24.973 INFO:tasks.workunit.client.0.smithi103.stdout:checking for uuid_compare... no 2024-09-11T00:25:25.044 INFO:tasks.workunit.client.0.smithi103.stdout:checking for uuid_compare in -luuid... yes 2024-09-11T00:25:25.188 INFO:tasks.workunit.client.0.smithi103.stdout:checking for xfs/xfs.h... yes 2024-09-11T00:25:25.266 INFO:tasks.workunit.client.0.smithi103.stdout:checking for xfs/libxfs.h... no 2024-09-11T00:25:25.952 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether xlog_assign_lsn is declared... no 2024-09-11T00:25:26.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:26 smithi103 ceph-mon[30578]: pgmap v469: 129 pgs: 129 active+clean; 46 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 1.25k op/s 2024-09-11T00:25:26.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:26 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1831193143' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:26.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:26 smithi183 ceph-mon[35934]: pgmap v469: 129 pgs: 129 active+clean; 46 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 1.25k op/s 2024-09-11T00:25:26.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:26 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1831193143' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:26 smithi117 ceph-mon[35525]: pgmap v469: 129 pgs: 129 active+clean; 46 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 1.25k op/s 2024-09-11T00:25:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:26 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1831193143' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:26.692 INFO:tasks.workunit.client.0.smithi103.stdout:checking for xfs/xqm.h... yes 2024-09-11T00:25:26.858 INFO:tasks.workunit.client.0.smithi103.stdout:checking xfsctl from xfs/xfs.h... ok 2024-09-11T00:25:28.054 INFO:tasks.workunit.client.0.smithi103.stdout:checking xfs/handle.h usability... yes 2024-09-11T00:25:28.083 INFO:tasks.workunit.client.0.smithi103.stdout:checking xfs/handle.h presence... yes 2024-09-11T00:25:28.083 INFO:tasks.workunit.client.0.smithi103.stdout:checking for xfs/handle.h... yes 2024-09-11T00:25:28.175 INFO:tasks.workunit.client.0.smithi103.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 -- 172.21.15.103:0/3545782874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 msgr2=0x7f14b4103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 --2- 172.21.15.103:0/3545782874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 0x7f14b4103ef0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009900 tx=0x7f14a402a410 comp rx=0 tx=0).stop 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 -- 172.21.15.103:0/3545782874 shutdown_connections 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 --2- 172.21.15.103:0/3545782874 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b410cf70 0x7f14b410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 --2- 172.21.15.103:0/3545782874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b4104430 0x7f14b410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 --2- 172.21.15.103:0/3545782874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 0x7f14b4103ef0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 -- 172.21.15.103:0/3545782874 >> 172.21.15.103:0/3545782874 conn(0x7f14b40fd5b0 msgr2=0x7f14b40ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 -- 172.21.15.103:0/3545782874 shutdown_connections 2024-09-11T00:25:28.203 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.706+0000 7f14b9400640 1 -- 172.21.15.103:0/3545782874 wait complete. 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.707+0000 7f14b9400640 1 Processor -- start 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b9400640 1 -- start start 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 0x7f14b41017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b4104430 0x7f14b4101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b9400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b410cf70 0x7f14b4102240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b9400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4112c30 con 0x7f14b4103b10 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b9400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4112ab0 con 0x7f14b4104430 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b9400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4112db0 con 0x7f14b410cf70 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 0x7f14b41017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 0x7f14b41017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43850/0 (socket says 172.21.15.103:43850) 2024-09-11T00:25:28.204 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b3e00640 1 -- 172.21.15.103:0/1831193143 learned_addr learned my addr 172.21.15.103:0/1831193143 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.708+0000 7f14b3400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b4104430 0x7f14b4101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.709+0000 7f14b8a00640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b410cf70 0x7f14b4102240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.709+0000 7f14b3400640 1 -- 172.21.15.103:0/1831193143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b410cf70 msgr2=0x7f14b4102240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.709+0000 7f14b3400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b410cf70 0x7f14b4102240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.709+0000 7f14b3400640 1 -- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 msgr2=0x7f14b41017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.709+0000 7f14b3400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 0x7f14b41017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.709+0000 7f14b3400640 1 -- 172.21.15.103:0/1831193143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b41a7440 con 0x7f14b4104430 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.709+0000 7f14b3400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b4104430 0x7f14b4101d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e770 tx=0x7f14a000ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.710+0000 7f14b0c00640 1 -- 172.21.15.103:0/1831193143 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a000ccc0 con 0x7f14b4104430 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.710+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b41a7730 con 0x7f14b4104430 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.710+0000 7f14b0c00640 1 -- 172.21.15.103:0/1831193143 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a000ee80 con 0x7f14b4104430 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.710+0000 7f14b0c00640 1 -- 172.21.15.103:0/1831193143 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003e40 con 0x7f14b4104430 2024-09-11T00:25:28.205 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.710+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b41a7cc0 con 0x7f14b4104430 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.711+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b4103ef0 con 0x7f14b4104430 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.714+0000 7f14b0c00640 1 -- 172.21.15.103:0/1831193143 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f14a0004000 con 0x7f14b4104430 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.715+0000 7f14b0c00640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14980b5390 0x7f14980b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.715+0000 7f14b0c00640 1 -- 172.21.15.103:0/1831193143 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f14a00e6280 con 0x7f14b4104430 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.715+0000 7f14b3e00640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14980b5390 0x7f14980b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.716+0000 7f14b0c00640 1 -- 172.21.15.103:0/1831193143 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f14a00acde0 con 0x7f14b4104430 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.716+0000 7f14b3e00640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14980b5390 0x7f14980b7850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14a40096c0 tx=0x7f14a4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.944+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14b41a8050 con 0x7f14b4104430 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.945+0000 7f14b0c00640 1 -- 172.21.15.103:0/1831193143 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11939 (secure 0 0 0) 0x7f14a00ac6e0 con 0x7f14b4104430 2024-09-11T00:25:28.206 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":33,"btime":"2024-09-11T00:25:18:176296+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:25:17.681542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25550":{"gid":25550,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3202362307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3202362307},{"type":"v1","addr":"172.21.15.183:6835","nonce":3202362307}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 33 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.947+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14980b5390 msgr2=0x7f14980b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.947+0000 7f14b9400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14980b5390 0x7f14980b7850 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f14a40096c0 tx=0x7f14a4005a20 comp rx=0 tx=0).stop 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.947+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b4104430 msgr2=0x7f14b4101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.947+0000 7f14b9400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b4104430 0x7f14b4101d00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14a000e770 tx=0x7f14a000ec30 comp rx=0 tx=0).stop 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.947+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 shutdown_connections 2024-09-11T00:25:28.207 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.947+0000 7f14b9400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f14980b5390 0x7f14980b7850 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.948+0000 7f14b9400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14b410cf70 0x7f14b4102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.948+0000 7f14b9400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14b4104430 0x7f14b4101d00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.948+0000 7f14b9400640 1 --2- 172.21.15.103:0/1831193143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14b4103b10 0x7f14b41017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:28.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.948+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 >> 172.21.15.103:0/1831193143 conn(0x7f14b40fd5b0 msgr2=0x7f14b40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:28.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.948+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 shutdown_connections 2024-09-11T00:25:28.208 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:25.948+0000 7f14b9400640 1 -- 172.21.15.103:0/1831193143 wait complete. 2024-09-11T00:25:28.244 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/acl.h usability... yes 2024-09-11T00:25:28.274 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/acl.h presence... yes 2024-09-11T00:25:28.274 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/acl.h... yes 2024-09-11T00:25:28.352 INFO:tasks.workunit.client.0.smithi103.stdout:checking acl/libacl.h usability... yes 2024-09-11T00:25:28.381 INFO:tasks.workunit.client.0.smithi103.stdout:checking acl/libacl.h presence... yes 2024-09-11T00:25:28.381 INFO:tasks.workunit.client.0.smithi103.stdout:checking for acl/libacl.h... yes 2024-09-11T00:25:28.450 INFO:tasks.workunit.client.0.smithi103.stdout:checking for acl_init in -lacl... yes 2024-09-11T00:25:28.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:28 smithi183 ceph-mon[35934]: pgmap v470: 129 pgs: 129 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.2 MiB/s wr, 1.17k op/s 2024-09-11T00:25:28.541 INFO:tasks.workunit.client.0.smithi103.stdout:checking gdbm-ndbm.h usability... no 2024-09-11T00:25:28.586 INFO:tasks.workunit.client.0.smithi103.stdout:checking gdbm-ndbm.h presence... no 2024-09-11T00:25:28.586 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gdbm-ndbm.h... no 2024-09-11T00:25:28.646 INFO:tasks.workunit.client.0.smithi103.stdout:checking gdbm/ndbm.h usability... no 2024-09-11T00:25:28.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:28 smithi103 ceph-mon[30578]: pgmap v470: 129 pgs: 129 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.2 MiB/s wr, 1.17k op/s 2024-09-11T00:25:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:28 smithi117 ceph-mon[35525]: pgmap v470: 129 pgs: 129 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 2.2 MiB/s wr, 1.17k op/s 2024-09-11T00:25:28.689 INFO:tasks.workunit.client.0.smithi103.stdout:checking gdbm/ndbm.h presence... no 2024-09-11T00:25:28.689 INFO:tasks.workunit.client.0.smithi103.stdout:checking for gdbm/ndbm.h... no 2024-09-11T00:25:28.755 INFO:tasks.workunit.client.0.smithi103.stdout:checking ndbm.h usability... no 2024-09-11T00:25:28.798 INFO:tasks.workunit.client.0.smithi103.stdout:checking ndbm.h presence... no 2024-09-11T00:25:28.798 INFO:tasks.workunit.client.0.smithi103.stdout:checking for ndbm.h... no 2024-09-11T00:25:28.865 INFO:tasks.workunit.client.0.smithi103.stdout:checking libaio.h usability... yes 2024-09-11T00:25:28.897 INFO:tasks.workunit.client.0.smithi103.stdout:checking libaio.h presence... yes 2024-09-11T00:25:28.897 INFO:tasks.workunit.client.0.smithi103.stdout:checking for libaio.h... yes 2024-09-11T00:25:28.963 INFO:tasks.workunit.client.0.smithi103.stdout:checking liburing.h usability... no 2024-09-11T00:25:29.006 INFO:tasks.workunit.client.0.smithi103.stdout:checking liburing.h presence... no 2024-09-11T00:25:29.006 INFO:tasks.workunit.client.0.smithi103.stdout:checking for liburing.h... no 2024-09-11T00:25:29.065 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/capability.h usability... no 2024-09-11T00:25:29.110 INFO:tasks.workunit.client.0.smithi103.stdout:checking sys/capability.h presence... no 2024-09-11T00:25:29.110 INFO:tasks.workunit.client.0.smithi103.stdout:checking for sys/capability.h... no 2024-09-11T00:25:29.168 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/fiemap.h usability... yes 2024-09-11T00:25:29.193 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/fiemap.h presence... yes 2024-09-11T00:25:29.193 INFO:tasks.workunit.client.0.smithi103.stdout:checking for linux/fiemap.h... yes 2024-09-11T00:25:29.261 INFO:tasks.workunit.client.0.smithi103.stdout:checking for fallocate... yes 2024-09-11T00:25:29.298 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-11T00:25:29.298 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-11T00:25:29.342 INFO:tasks.workunit.client.0.smithi103.stdout:checking for open_by_handle_at... yes 2024-09-11T00:25:29.402 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/fs.h usability... yes 2024-09-11T00:25:29.428 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/fs.h presence... yes 2024-09-11T00:25:29.428 INFO:tasks.workunit.client.0.smithi103.stdout:checking for linux/fs.h... yes 2024-09-11T00:25:29.502 INFO:tasks.workunit.client.0.smithi103.stdout:checking btrfsutil.h usability... no 2024-09-11T00:25:29.543 INFO:tasks.workunit.client.0.smithi103.stdout:checking btrfsutil.h presence... no 2024-09-11T00:25:29.544 INFO:tasks.workunit.client.0.smithi103.stdout:checking for btrfsutil.h... no 2024-09-11T00:25:29.613 INFO:tasks.workunit.client.0.smithi103.stdout:checking for copy_file_range... yes 2024-09-11T00:25:29.691 INFO:tasks.workunit.client.0.smithi103.stdout:checking for SEEK_DATA... yes 2024-09-11T00:25:29.729 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:29.788 INFO:tasks.workunit.client.0.smithi103.stdout:checking for BMV_OF_SHARED... yes 2024-09-11T00:25:29.929 INFO:tasks.workunit.client.0.smithi103.stdout:checking for nftw... yes 2024-09-11T00:25:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 -- 172.21.15.103:0/2270549545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 msgr2=0x7f46b010d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 --2- 172.21.15.103:0/2270549545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 0x7f46b010d030 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009cd0 tx=0x7f46a802a8e0 comp rx=0 tx=0).stop 2024-09-11T00:25:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 -- 172.21.15.103:0/2270549545 shutdown_connections 2024-09-11T00:25:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 --2- 172.21.15.103:0/2270549545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 0x7f46b010d030 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 --2- 172.21.15.103:0/2270549545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 0x7f46b0106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 --2- 172.21.15.103:0/2270549545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b0104ba0 0x7f46b0104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 -- 172.21.15.103:0/2270549545 >> 172.21.15.103:0/2270549545 conn(0x7f46b0100460 msgr2=0x7f46b0102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 -- 172.21.15.103:0/2270549545 shutdown_connections 2024-09-11T00:25:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.990+0000 7f46b7a00640 1 -- 172.21.15.103:0/2270549545 wait complete. 2024-09-11T00:25:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.991+0000 7f46b7a00640 1 Processor -- start 2024-09-11T00:25:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.991+0000 7f46b7a00640 1 -- start start 2024-09-11T00:25:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b7a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b0104ba0 0x7f46b019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b7a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 0x7f46b019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b7a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 0x7f46b019c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b7a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46b0110860 con 0x7f46b01066b0 2024-09-11T00:25:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b7a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01106e0 con 0x7f46b0105cf0 2024-09-11T00:25:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b7a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01109e0 con 0x7f46b0104ba0 2024-09-11T00:25:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 0x7f46b019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 0x7f46b019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45270/0 (socket says 172.21.15.103:45270) 2024-09-11T00:25:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b5a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 0x7f46b019c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 -- 172.21.15.103:0/605325113 learned_addr learned my addr 172.21.15.103:0/605325113 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46b5000640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b0104ba0 0x7f46b019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 -- 172.21.15.103:0/605325113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b0104ba0 msgr2=0x7f46b019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b0104ba0 0x7f46b019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 -- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 msgr2=0x7f46b019c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 0x7f46b019c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.992+0000 7f46afe00640 1 -- 172.21.15.103:0/605325113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b019ca40 con 0x7f46b0105cf0 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.993+0000 7f46afe00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 0x7f46b019b550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f469c00ea60 tx=0x7f469c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.993+0000 7f46ad600640 1 -- 172.21.15.103:0/605325113 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c00cde0 con 0x7f46b0105cf0 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.993+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b019cd30 con 0x7f46b0105cf0 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.993+0000 7f46ad600640 1 -- 172.21.15.103:0/605325113 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c0035f0 con 0x7f46b0105cf0 2024-09-11T00:25:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.993+0000 7f46ad600640 1 -- 172.21.15.103:0/605325113 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c010690 con 0x7f46b0105cf0 2024-09-11T00:25:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.993+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b01a3400 con 0x7f46b0105cf0 2024-09-11T00:25:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.997+0000 7f46ba7f2740 1 Processor -- start 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.997+0000 7f46ba7f2740 1 -- start start 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.997+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fb790 0x5652522fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fe0c0 0x5652523004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x565252302fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56525226e6b0 con 0x565252300b00 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x56525225dae0 con 0x5652522fb790 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5652522618f0 con 0x5652522fe0c0 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46ad600640 1 -- 172.21.15.103:0/605325113 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f469c004000 con 0x7f46b0105cf0 2024-09-11T00:25:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fe0c0 0x5652523004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fb790 0x5652522fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fe0c0 0x5652523004b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:35462/0 (socket says 172.21.15.103:35462) 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fb790 0x5652522fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45282/0 (socket says 172.21.15.103:45282) 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 -- 172.21.15.103:0/2960628848 learned_addr learned my addr 172.21.15.103:0/2960628848 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46afe00640 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x565252302fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 -- 172.21.15.103:0/2960628848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fb790 msgr2=0x5652522fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fb790 0x5652522fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 -- 172.21.15.103:0/2960628848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 msgr2=0x565252302fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x565252302fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.998+0000 7f46b5000640 1 -- 172.21.15.103:0/2960628848 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56525222a780 con 0x5652522fe0c0 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.999+0000 7f46b5000640 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fe0c0 0x5652523004b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009d20 tx=0x7f46a4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ccc5b1a05b25b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.999+0000 7f46a2000640 1 -- 172.21.15.103:0/2960628848 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a401e070 con 0x5652522fe0c0 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.999+0000 7f46a2000640 1 -- 172.21.15.103:0/2960628848 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a4019ae0 con 0x5652522fe0c0 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.999+0000 7f46a2000640 1 -- 172.21.15.103:0/2960628848 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a400ea80 con 0x5652522fe0c0 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:29.999+0000 7f46ad600640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46980b52e0 0x7f46980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ad600640 1 -- 172.21.15.103:0/605325113 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f469c0e62a0 con 0x7f46b0105cf0 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46b5a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46980b52e0 0x7f46980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2960628848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fe0c0 msgr2=0x5652523004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fe0c0 0x5652523004b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009d20 tx=0x7f46a4011810 comp rx=0 tx=0).stop 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2960628848 shutdown_connections 2024-09-11T00:25:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x565252302fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fe0c0 0x5652523004b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2960628848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fb790 0x5652522fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2960628848 >> 172.21.15.103:0/2960628848 conn(0x5652522f68e0 msgr2=0x5652522f7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2960628848 shutdown_connections 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2960628848 wait complete. 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.000+0000 7f46b5a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46980b52e0 0x7f46980b77a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006010 tx=0x7f46a8031850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 -- start start 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 Processor -- start 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x565252304ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x565252305540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x565252306110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565252195ab0 con 0x5652522fb790 2024-09-11T00:25:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565252204570 con 0x5652522fe0c0 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565252303ae0 con 0x565252300b00 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.001+0000 7f46ba7f2740 10 client.0 _ll_register_callbacks cb 0x565252213100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x565252305540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x565252305540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45288/0 (socket says 172.21.15.103:45288) 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 -- 172.21.15.103:0/2274868353 learned_addr learned my addr 172.21.15.103:0/2274868353 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46b5000640 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x565252304ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46b5a00640 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x565252306110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 -- 172.21.15.103:0/2274868353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 msgr2=0x565252306110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x565252306110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 -- 172.21.15.103:0/2274868353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 msgr2=0x565252304ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x565252304ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 -- 172.21.15.103:0/2274868353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565251fbf790 con 0x5652522fe0c0 2024-09-11T00:25:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46afe00640 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x565252305540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f469c0ac600 tx=0x7f469c003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d7d79421b5a990 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f4696a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f4696a00640 1 -- 172.21.15.103:0/2274868353 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0abde0 con 0x5652522fe0c0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a4019ae0 con 0x5652522fe0c0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.002+0000 7f4696a00640 1 -- 172.21.15.103:0/2274868353 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f469c0ab8d0 con 0x5652522fe0c0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f4696a00640 1 -- 172.21.15.103:0/2274868353 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f469c0e9910 con 0x5652522fe0c0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 10 client.25568 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x565252038620 con 0x5652522fe0c0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f4696a00640 1 -- 172.21.15.103:0/2274868353 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a4019ae0 con 0x5652522fe0c0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f4696a00640 1 -- 172.21.15.103:0/2274868353 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f469c0a7fc0 con 0x5652522fe0c0 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 10 client.25568 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 20 client.25568 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565252310760 0x565252312bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 4 client.25568 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.003+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565252270600 con 0x565252310760 2024-09-11T00:25:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.004+0000 7f46b5000640 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565252310760 0x565252312bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.004+0000 7f46b5000640 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565252310760 0x565252312bd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e573fc10ad9ea37 server_cookie=42a2bcee89b9f364 in_seq=0 out_seq=0 2024-09-11T00:25:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.004+0000 7f4696a00640 10 client.25568 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f4696a00640 1 -- 172.21.15.103:0/2274868353 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565252270600 con 0x565252310760 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f4696a00640 10 client.25568 handle_command_reply: tid=0 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 client.25568 shutdown 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 msgr2=0x565252305540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x565252305540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f469c0ac600 tx=0x7f469c003cf0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 shutdown_connections 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565252310760 0x565252312bd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x565252306110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x565252305540 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f4696a00640 0 client.25568 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/2274868353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x565252304ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 >> 172.21.15.103:0/2274868353 conn(0x5652522f68e0 msgr2=0x5652522f5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.005+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 shutdown_connections 2024-09-11T00:25:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.006+0000 7f46ba7f2740 1 -- 172.21.15.103:0/2274868353 wait complete. 2024-09-11T00:25:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.006+0000 7f46ba7f2740 20 client.25568 trim_cache size 0 max 16384 2024-09-11T00:25:30.019 INFO:tasks.workunit.client.0.smithi103.stdout:checking for RLIMIT_NOFILE... yes 2024-09-11T00:25:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.022+0000 7f46ba7f2740 1 Processor -- start 2024-09-11T00:25:30.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.022+0000 7f46ba7f2740 1 -- start start 2024-09-11T00:25:30.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x5652522f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x5652522f9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x5652522f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01b0160 con 0x5652522fb790 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01ad970 con 0x5652522fe0c0 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01ab7c0 con 0x565252300b00 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46b5000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x5652522f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46afe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x5652522f9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46b5000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x5652522f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43914/0 (socket says 172.21.15.103:43914) 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46b5000640 1 -- 172.21.15.103:0/1238043659 learned_addr learned my addr 172.21.15.103:0/1238043659 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.023+0000 7f46b5a00640 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x5652522f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46afe00640 1 -- 172.21.15.103:0/1238043659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 msgr2=0x5652522f9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:30.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46afe00640 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x5652522f9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46afe00640 1 -- 172.21.15.103:0/1238043659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 msgr2=0x5652522f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46afe00640 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x5652522f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46afe00640 1 -- 172.21.15.103:0/1238043659 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652522fdef0 con 0x5652522fe0c0 2024-09-11T00:25:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46b5000640 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x5652522f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46afe00640 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x5652522f9930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f469c00c760 tx=0x7f469c0e6170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5523da2274fe1367 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46a1600640 1 -- 172.21.15.103:0/1238043659 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0b5c50 con 0x5652522fe0c0 2024-09-11T00:25:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46a1600640 1 -- 172.21.15.103:0/1238043659 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c0ac180 con 0x5652522fe0c0 2024-09-11T00:25:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.024+0000 7f46a1600640 1 -- 172.21.15.103:0/1238043659 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0ac440 con 0x5652522fe0c0 2024-09-11T00:25:30.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 -- 172.21.15.103:0/1238043659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 msgr2=0x5652522f9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x5652522f9930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f469c00c760 tx=0x7f469c0e6170 comp rx=0 tx=0).stop 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 -- 172.21.15.103:0/1238043659 shutdown_connections 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565252300b00 0x5652522f9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x5652522f9930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/1238043659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652522fb790 0x5652522f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 -- 172.21.15.103:0/1238043659 >> 172.21.15.103:0/1238043659 conn(0x5652522f68e0 msgr2=0x5652522fac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 -- 172.21.15.103:0/1238043659 shutdown_connections 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.025+0000 7f46ba7f2740 1 -- 172.21.15.103:0/1238043659 wait complete. 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.026+0000 7f46ba7f2740 1 -- start start 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.026+0000 7f46ba7f2740 1 Processor -- start 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.026+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fb790 0x56525230cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.026+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x56525230b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.026+0000 7f46ba7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x56525230bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01abe30 con 0x565252300b00 2024-09-11T00:25:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01ad2a0 con 0x5652522fe0c0 2024-09-11T00:25:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46ba7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f46b01ac500 con 0x5652522fb790 2024-09-11T00:25:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46ba7f2740 10 client.0 _ll_register_callbacks cb 0x7f469c004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fb790 0x56525230cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x56525230bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fb790 0x56525230cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:35480/0 (socket says 172.21.15.103:35480) 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 -- 172.21.15.103:0/3400318922 learned_addr learned my addr 172.21.15.103:0/3400318922 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 -- 172.21.15.103:0/3400318922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 msgr2=0x56525230b840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x56525230b840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 -- 172.21.15.103:0/3400318922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 msgr2=0x56525230bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x56525230bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.027+0000 7f46b5000640 1 -- 172.21.15.103:0/3400318922 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56525230a360 con 0x5652522fb790 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f46b5000640 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fb790 0x56525230cb40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005890 tx=0x7f46a40096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e247d0ffa8b8e299 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f4696a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f4696a00640 1 -- 172.21.15.103:0/3400318922 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a401e040 con 0x5652522fb790 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f469c0ac180 con 0x5652522fb790 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f4696a00640 1 -- 172.21.15.103:0/3400318922 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f46a40052b0 con 0x5652522fb790 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f4696a00640 1 -- 172.21.15.103:0/3400318922 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f46a4005c50 con 0x5652522fb790 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f46ba7f2740 10 client.25603 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.028+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56525237d990 con 0x5652522fb790 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f4696a00640 1 -- 172.21.15.103:0/3400318922 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c0ac180 con 0x5652522fb790 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f4696a00640 1 -- 172.21.15.103:0/3400318922 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f46a4032070 con 0x5652522fb790 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f46ba7f2740 10 client.25603 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f46ba7f2740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f46ba7f2740 20 client.25603 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56525237e200 0x565252380620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:30.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f46ba7f2740 4 client.25603 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56525226c0b0 con 0x56525237e200 2024-09-11T00:25:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.029+0000 7f46afe00640 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56525237e200 0x565252380620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.030+0000 7f46afe00640 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56525237e200 0x565252380620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d7d7bb27c93e210 server_cookie=8528ea326dadde3b in_seq=0 out_seq=0 2024-09-11T00:25:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.030+0000 7f4696a00640 10 client.25603 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:30.097 INFO:tasks.workunit.client.0.smithi103.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-09-11T00:25:30.290 INFO:tasks.workunit.client.0.smithi103.stdout:checking for FICLONE... yes 2024-09-11T00:25:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.301+0000 7f4696a00640 1 -- 172.21.15.103:0/3400318922 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56525226c0b0 con 0x56525237e200 2024-09-11T00:25:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.301+0000 7f4696a00640 10 client.25603 handle_command_reply: tid=0 2024-09-11T00:25:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.301+0000 7f46ba7f2740 1 client.25603 shutdown 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.301+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fb790 msgr2=0x56525230cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.301+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fb790 0x56525230cb40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005890 tx=0x7f46a40096c0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 shutdown_connections 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565252300b00 0x56525230bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652522fe0c0 0x56525230b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x56525237e200 0x565252380620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 --2- 172.21.15.103:0/3400318922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5652522fb790 0x56525230cb40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f4696a00640 0 client.25603 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 >> 172.21.15.103:0/3400318922 conn(0x5652522f68e0 msgr2=0x5652522fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 shutdown_connections 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 1 -- 172.21.15.103:0/3400318922 wait complete. 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.302+0000 7f46ba7f2740 20 client.25603 trim_cache size 0 max 16384 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:25:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:25:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46980b52e0 msgr2=0x7f46980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46980b52e0 0x7f46980b77a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006010 tx=0x7f46a8031850 comp rx=0 tx=0).stop 2024-09-11T00:25:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 msgr2=0x7f46b019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 0x7f46b019b550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f469c00ea60 tx=0x7f469c00ef20 comp rx=0 tx=0).stop 2024-09-11T00:25:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 shutdown_connections 2024-09-11T00:25:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f46980b52e0 0x7f46980b77a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46b01066b0 0x7f46b019c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46b0105cf0 0x7f46b019b550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 --2- 172.21.15.103:0/605325113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f46b0104ba0 0x7f46b019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 >> 172.21.15.103:0/605325113 conn(0x7f46b0100460 msgr2=0x7f46b0108ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 shutdown_connections 2024-09-11T00:25:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:30.303+0000 7f46b7a00640 1 -- 172.21.15.103:0/605325113 wait complete. 2024-09-11T00:25:30.380 INFO:tasks.workunit.client.0.smithi103.stdout:checking for renameat2... yes 2024-09-11T00:25:30.485 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:30 smithi103 ceph-mon[30578]: pgmap v471: 129 pgs: 129 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 1.8 MiB/s wr, 1k op/s 2024-09-11T00:25:30.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:30 smithi183 ceph-mon[35934]: pgmap v471: 129 pgs: 129 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 1.8 MiB/s wr, 1k op/s 2024-09-11T00:25:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:30 smithi117 ceph-mon[35525]: pgmap v471: 129 pgs: 129 active+clean; 46 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 1.8 MiB/s wr, 1k op/s 2024-09-11T00:25:30.849 INFO:tasks.workunit.client.0.smithi103.stdout:checking for reallocarray... yes 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 -- 172.21.15.103:0/216796854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075540 msgr2=0x7f7910075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 --2- 172.21.15.103:0/216796854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075540 0x7f7910075920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7900009900 tx=0x7f790002a410 comp rx=0 tx=0).stop 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 -- 172.21.15.103:0/216796854 shutdown_connections 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 --2- 172.21.15.103:0/216796854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910113960 0x7f7910115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 --2- 172.21.15.103:0/216796854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7910075e60 0x7f7910113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 --2- 172.21.15.103:0/216796854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075540 0x7f7910075920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 -- 172.21.15.103:0/216796854 >> 172.21.15.103:0/216796854 conn(0x7f79100fe6d0 msgr2=0x7f7910100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 -- 172.21.15.103:0/216796854 shutdown_connections 2024-09-11T00:25:31.054 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.606+0000 7f790fe00640 1 -- 172.21.15.103:0/216796854 wait complete. 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.607+0000 7f790fe00640 1 Processor -- start 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.607+0000 7f790fe00640 1 -- start start 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910075540 0x7f791019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075e60 0x7f791019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7910113960 0x7f79101a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7910119200 con 0x7f7910075e60 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7910119080 con 0x7f7910075540 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7910119380 con 0x7f7910113960 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075e60 0x7f791019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075e60 0x7f791019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43944/0 (socket says 172.21.15.103:43944) 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 -- 172.21.15.103:0/394938663 learned_addr learned my addr 172.21.15.103:0/394938663 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790ea00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910075540 0x7f791019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.055 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790f400640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7910113960 0x7f79101a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 -- 172.21.15.103:0/394938663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7910113960 msgr2=0x7f79101a4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7910113960 0x7f79101a4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 -- 172.21.15.103:0/394938663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910075540 msgr2=0x7f791019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910075540 0x7f791019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f7907e00640 1 -- 172.21.15.103:0/394938663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a5780 con 0x7f7910075e60 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.608+0000 7f790f400640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7910113960 0x7f79101a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.609+0000 7f7907e00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075e60 0x7f791019fc10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f78f800e9f0 tx=0x7f78f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.609+0000 7f7907400640 1 -- 172.21.15.103:0/394938663 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f800cd80 con 0x7f7910075e60 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.609+0000 7f7907400640 1 -- 172.21.15.103:0/394938663 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f80054d0 con 0x7f7910075e60 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.609+0000 7f7907400640 1 -- 172.21.15.103:0/394938663 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f8010690 con 0x7f7910075e60 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.609+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a5a70 con 0x7f7910075e60 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.609+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101ac130 con 0x7f7910075e60 2024-09-11T00:25:31.056 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.612+0000 7f791860e740 1 Processor -- start 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.612+0000 7f791860e740 1 -- start start 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f7907400640 1 -- 172.21.15.103:0/394938663 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f78f8004e90 con 0x7f7910075e60 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f791860e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f791860e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f791860e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f791860e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6bc634ca0 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f791860e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6bc6240d0 con 0x55b6bc6cbba0 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f791860e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6bc627ee0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f790f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f790f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6bf6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43958/0 (socket says 172.21.15.103:43958) 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f790f400640 1 -- 172.21.15.103:0/3019565968 learned_addr learned my addr 172.21.15.103:0/3019565968 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:31.057 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f790ea00640 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f7907e00640 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.613+0000 7f790f400640 1 -- 172.21.15.103:0/3019565968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 msgr2=0x55b6bc6cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f790f400640 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f790f400640 1 -- 172.21.15.103:0/3019565968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6cbba0 msgr2=0x55b6bc6cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f790f400640 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f790f400640 1 -- 172.21.15.103:0/3019565968 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6bc5d6400 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f7907e00640 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f790ea00640 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f790f400640 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6bf6d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7908010480 tx=0x7f7908009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41690174a41a52de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f78ffe00640 1 -- 172.21.15.103:0/3019565968 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908011760 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f78ffe00640 1 -- 172.21.15.103:0/3019565968 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908011db0 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f78ffe00640 1 -- 172.21.15.103:0/3019565968 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790801a380 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.058 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.614+0000 7f7907400640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f78f40b5390 0x7f78f40b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f7907400640 1 -- 172.21.15.103:0/394938663 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f78f80e63e0 con 0x7f7910075e60 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f790ea00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f78f40b5390 0x7f78f40b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 -- 172.21.15.103:0/3019565968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6bf2f0 msgr2=0x55b6bc6bf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6bf6d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7908010480 tx=0x7f7908009f90 comp rx=0 tx=0).stop 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f790ea00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f78f40b5390 0x7f78f40b7850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7900031440 tx=0x7f7900005b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 -- 172.21.15.103:0/3019565968 shutdown_connections 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 --2- 172.21.15.103:0/3019565968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6bf6d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 -- 172.21.15.103:0/3019565968 >> 172.21.15.103:0/3019565968 conn(0x55b6bc6bd120 msgr2=0x55b6bc6be2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 -- 172.21.15.103:0/3019565968 shutdown_connections 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.615+0000 7f791860e740 1 -- 172.21.15.103:0/3019565968 wait complete. 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.616+0000 7f791860e740 1 -- start start 2024-09-11T00:25:31.059 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.616+0000 7f791860e740 1 Processor -- start 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.616+0000 7f791860e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.616+0000 7f791860e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.616+0000 7f791860e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f791860e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b6bc4857a0 con 0x55b6bc6cbba0 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f791860e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b6bc5c9a10 con 0x55b6bc6bfc10 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f791860e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b6bc6ce820 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f791860e740 10 client.0 _ll_register_callbacks cb 0x55b6bc5d9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45328/0 (socket says 172.21.15.103:45328) 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 -- 172.21.15.103:0/1730265294 learned_addr learned my addr 172.21.15.103:0/1730265294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f790ea00640 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 -- 172.21.15.103:0/1730265294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 msgr2=0x55b6bc6cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:31.060 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 -- 172.21.15.103:0/1730265294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 msgr2=0x55b6bc6c8f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c8f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 -- 172.21.15.103:0/1730265294 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6bc391b90 con 0x55b6bc6bfc10 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f790ea00640 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f7907e00640 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6c88e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78f80ac550 tx=0x7f78f8004a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903f53ca1a30f5e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.617+0000 7f78fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f78fcc00640 1 -- 172.21.15.103:0/1730265294 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80ac1a0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f78fcc00640 1 -- 172.21.15.103:0/1730265294 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f78f80e60d0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7908011db0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f78fcc00640 1 -- 172.21.15.103:0/1730265294 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f78f80abfc0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 10 client.25583 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b6bc6da180 con 0x55b6bc6bfc10 2024-09-11T00:25:31.061 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f78fcc00640 1 -- 172.21.15.103:0/1730265294 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908011db0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f78fcc00640 1 -- 172.21.15.103:0/1730265294 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f78f80a8ac0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 10 client.25583 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 10 client.25583 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 20 client.25583 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc6db570 0x55b6bc6dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 4 client.25583 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.618+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6bc636bf0 con 0x55b6bc6db570 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.619+0000 7f790ea00640 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc6db570 0x55b6bc6dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.619+0000 7f790ea00640 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc6db570 0x55b6bc6dd9d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b657383a56fef8a3 server_cookie=ae4293b970d9600f in_seq=0 out_seq=0 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.619+0000 7f78fcc00640 10 client.25583 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f78fcc00640 1 -- 172.21.15.103:0/1730265294 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6bc636bf0 con 0x55b6bc6db570 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f78fcc00640 10 client.25583 handle_command_reply: tid=0 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 client.25583 shutdown 2024-09-11T00:25:31.062 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 msgr2=0x55b6bc6c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6c88e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78f80ac550 tx=0x7f78f8004a00 comp rx=0 tx=0).stop 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 shutdown_connections 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6c88e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc6db570 0x55b6bc6dd9d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 --2- 172.21.15.103:0/1730265294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f78fcc00640 0 client.25583 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 >> 172.21.15.103:0/1730265294 conn(0x55b6bc6bd120 msgr2=0x55b6bc6c2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 shutdown_connections 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 1 -- 172.21.15.103:0/1730265294 wait complete. 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.620+0000 7f791860e740 20 client.25583 trim_cache size 0 max 16384 2024-09-11T00:25:31.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.638+0000 7f791860e740 1 Processor -- start 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.638+0000 7f791860e740 1 -- start start 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f791860e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f791860e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f791860e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f791860e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b8850 con 0x55b6bc6cbba0 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f791860e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b6060 con 0x55b6bc6bfc10 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f791860e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b3eb0 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f790f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f7907e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f790f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43962/0 (socket says 172.21.15.103:43962) 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f790f400640 1 -- 172.21.15.103:0/2432851386 learned_addr learned my addr 172.21.15.103:0/2432851386 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f790ea00640 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f7907e00640 1 -- 172.21.15.103:0/2432851386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 msgr2=0x55b6bc6d7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f7907e00640 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f7907e00640 1 -- 172.21.15.103:0/2432851386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 msgr2=0x55b6bc6c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f7907e00640 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.639+0000 7f7907e00640 1 -- 172.21.15.103:0/2432851386 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6bc6bfa40 con 0x55b6bc6bfc10 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.640+0000 7f7907e00640 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6bca10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78f80a8e20 tx=0x7f78f80057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3d0b5ee47aadff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.640+0000 7f78ff400640 1 -- 172.21.15.103:0/2432851386 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80ac1a0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.640+0000 7f78ff400640 1 -- 172.21.15.103:0/2432851386 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f80ac360 con 0x55b6bc6bfc10 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.640+0000 7f78ff400640 1 -- 172.21.15.103:0/2432851386 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f80a86f0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.640+0000 7f791860e740 1 -- 172.21.15.103:0/2432851386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 msgr2=0x55b6bc6bca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.640+0000 7f791860e740 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6bca10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78f80a8e20 tx=0x7f78f80057c0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 -- 172.21.15.103:0/2432851386 shutdown_connections 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc6c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6bca10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 --2- 172.21.15.103:0/2432851386 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 -- 172.21.15.103:0/2432851386 >> 172.21.15.103:0/2432851386 conn(0x55b6bc6bd120 msgr2=0x55b6bc6bc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 -- 172.21.15.103:0/2432851386 shutdown_connections 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 -- 172.21.15.103:0/2432851386 wait complete. 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 -- start start 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.641+0000 7f791860e740 1 Processor -- start 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f791860e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f791860e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f791860e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc7484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f791860e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b4520 con 0x55b6bc6cbba0 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f791860e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b5990 con 0x55b6bc6bf2f0 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f791860e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b4bf0 con 0x55b6bc6bfc10 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f791860e740 10 client.0 _ll_register_callbacks cb 0x7f78f80a6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc7484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc7484b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43970/0 (socket says 172.21.15.103:43970) 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 -- 172.21.15.103:0/200032377 learned_addr learned my addr 172.21.15.103:0/200032377 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790ea00640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f7907e00640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 -- 172.21.15.103:0/200032377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 msgr2=0x55b6bc6d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 -- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bf2f0 msgr2=0x55b6bc6d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790f400640 1 -- 172.21.15.103:0/200032377 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6bc6c9280 con 0x55b6bc6cbba0 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.642+0000 7f790ea00640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f790f400640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc7484b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7908004f80 tx=0x7f7908009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84aa75fa93a79d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f78fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f78fcc00640 1 -- 172.21.15.103:0/200032377 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908011830 con 0x55b6bc6cbba0 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f78fcc00640 1 -- 172.21.15.103:0/200032377 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f79080119d0 con 0x55b6bc6cbba0 2024-09-11T00:25:31.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78f80ac360 con 0x55b6bc6cbba0 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f78fcc00640 1 -- 172.21.15.103:0/200032377 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7908011bd0 con 0x55b6bc6cbba0 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f791860e740 10 client.15573 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.643+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b6bc749b00 con 0x55b6bc6cbba0 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f78fcc00640 1 -- 172.21.15.103:0/200032377 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f80ac360 con 0x55b6bc6cbba0 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f78fcc00640 1 -- 172.21.15.103:0/200032377 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f7908004fb0 con 0x55b6bc6cbba0 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f791860e740 10 client.15573 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f791860e740 10 client.15573 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f791860e740 20 client.15573 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f791860e740 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc74a070 0x55b6bc74c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f791860e740 4 client.15573 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x55b6bc6326a0 con 0x55b6bc74a070 2024-09-11T00:25:31.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.644+0000 7f790ea00640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc74a070 0x55b6bc74c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:31.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.645+0000 7f790ea00640 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc74a070 0x55b6bc74c490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a57b88e00cc5fb8 server_cookie=61d50aedcc2e5dbf in_seq=0 out_seq=0 2024-09-11T00:25:31.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:30.645+0000 7f78fcc00640 10 client.15573 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:32.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:32 smithi183 ceph-mon[35934]: pgmap v472: 129 pgs: 129 active+clean; 47 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.2 MiB/s wr, 647 op/s 2024-09-11T00:25:32.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:32 smithi103 ceph-mon[30578]: pgmap v472: 129 pgs: 129 active+clean; 47 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.2 MiB/s wr, 647 op/s 2024-09-11T00:25:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:32 smithi117 ceph-mon[35525]: pgmap v472: 129 pgs: 129 active+clean; 47 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 1.2 MiB/s wr, 647 op/s 2024-09-11T00:25:34.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:34 smithi183 ceph-mon[35934]: pgmap v473: 129 pgs: 129 active+clean; 47 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 227 KiB/s wr, 187 op/s 2024-09-11T00:25:34.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:34 smithi103 ceph-mon[30578]: pgmap v473: 129 pgs: 129 active+clean; 47 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 227 KiB/s wr, 187 op/s 2024-09-11T00:25:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:34 smithi117 ceph-mon[35525]: pgmap v473: 129 pgs: 129 active+clean; 47 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 227 KiB/s wr, 187 op/s 2024-09-11T00:25:35.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.246+0000 7f78fcc00640 1 -- 172.21.15.103:0/200032377 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b6bc6326a0 con 0x55b6bc74a070 2024-09-11T00:25:35.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.246+0000 7f78fcc00640 10 client.15573 handle_command_reply: tid=0 2024-09-11T00:25:35.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.246+0000 7f791860e740 1 client.15573 shutdown 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 msgr2=0x55b6bc7484b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc7484b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7908004f80 tx=0x7f7908009940 comp rx=0 tx=0).stop 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 shutdown_connections 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b6bc6cbba0 0x55b6bc7484b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b6bc74a070 0x55b6bc74c490 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b6bc6bfc10 0x55b6bc6d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 --2- 172.21.15.103:0/200032377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b6bc6bf2f0 0x55b6bc6d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 >> 172.21.15.103:0/200032377 conn(0x55b6bc6bd120 msgr2=0x55b6bc6d6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f78fcc00640 0 client.15573 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 shutdown_connections 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 1 -- 172.21.15.103:0/200032377 wait complete. 2024-09-11T00:25:35.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.247+0000 7f791860e740 20 client.15573 trim_cache size 0 max 16384 2024-09-11T00:25:35.251 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:25:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 33, 2024-09-11T00:25:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:25:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 6, 2024-09-11T00:25:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:25:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "b20c3cfc": { 2024-09-11T00:25:35.251 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 6, 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.5, 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:25:35.252 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:35.253 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:35.253 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:25:35.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f78f40b5390 msgr2=0x7f78f40b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:35.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f78f40b5390 0x7f78f40b7850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7900031440 tx=0x7f7900005b90 comp rx=0 tx=0).stop 2024-09-11T00:25:35.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075e60 msgr2=0x7f791019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:35.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075e60 0x7f791019fc10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f78f800e9f0 tx=0x7f78f800edf0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 shutdown_connections 2024-09-11T00:25:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f78f40b5390 0x7f78f40b7850 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7910113960 0x7f79101a4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7910075e60 0x7f791019fc10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 --2- 172.21.15.103:0/394938663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7910075540 0x7f791019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 >> 172.21.15.103:0/394938663 conn(0x7f79100fe6d0 msgr2=0x7f79100ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 shutdown_connections 2024-09-11T00:25:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:35.248+0000 7f790fe00640 1 -- 172.21.15.103:0/394938663 wait complete. 2024-09-11T00:25:36.424 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b20c3cfc', quiesce took 12.1 seconds. Will release after: 8.4 2024-09-11T00:25:36.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:36 smithi183 ceph-mon[35934]: pgmap v474: 129 pgs: 129 active+clean; 44 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 296 KiB/s wr, 143 op/s 2024-09-11T00:25:36.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:36 smithi103 ceph-mon[30578]: pgmap v474: 129 pgs: 129 active+clean; 44 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 296 KiB/s wr, 143 op/s 2024-09-11T00:25:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:36 smithi117 ceph-mon[35525]: pgmap v474: 129 pgs: 129 active+clean; 44 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 296 KiB/s wr, 143 op/s 2024-09-11T00:25:38.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:38 smithi183 ceph-mon[35934]: pgmap v475: 129 pgs: 129 active+clean; 44 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 220 KiB/s wr, 144 op/s 2024-09-11T00:25:38.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:38 smithi103 ceph-mon[30578]: pgmap v475: 129 pgs: 129 active+clean; 44 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 220 KiB/s wr, 144 op/s 2024-09-11T00:25:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:38 smithi117 ceph-mon[35525]: pgmap v475: 129 pgs: 129 active+clean; 44 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 220 KiB/s wr, 144 op/s 2024-09-11T00:25:40.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:40 smithi183 ceph-mon[35934]: pgmap v476: 129 pgs: 129 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 378 KiB/s wr, 178 op/s 2024-09-11T00:25:40.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:25:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:40 smithi103 ceph-mon[30578]: pgmap v476: 129 pgs: 129 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 378 KiB/s wr, 178 op/s 2024-09-11T00:25:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:25:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:40 smithi117 ceph-mon[35525]: pgmap v476: 129 pgs: 129 active+clean; 46 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 378 KiB/s wr, 178 op/s 2024-09-11T00:25:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:25:41.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:25:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:25:41.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:25:41.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:25:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:25:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:25:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:25:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:25:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:25:42.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:42 smithi183 ceph-mon[35934]: pgmap v477: 129 pgs: 129 active+clean; 47 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 504 KiB/s wr, 192 op/s 2024-09-11T00:25:42.571 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:42 smithi103 ceph-mon[30578]: pgmap v477: 129 pgs: 129 active+clean; 47 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 504 KiB/s wr, 192 op/s 2024-09-11T00:25:42.588 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:42 smithi117 ceph-mon[35525]: pgmap v477: 129 pgs: 129 active+clean; 47 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 504 KiB/s wr, 192 op/s 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 -- 172.21.15.103:0/2594863897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41049b0 msgr2=0x7f98e4104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 --2- 172.21.15.103:0/2594863897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41049b0 0x7f98e4104db0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009930 tx=0x7f98d802a420 comp rx=0 tx=0).stop 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 -- 172.21.15.103:0/2594863897 shutdown_connections 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 --2- 172.21.15.103:0/2594863897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e41064c0 0x7f98e410ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 --2- 172.21.15.103:0/2594863897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e4105b00 0x7f98e4105f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 --2- 172.21.15.103:0/2594863897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41049b0 0x7f98e4104db0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 -- 172.21.15.103:0/2594863897 >> 172.21.15.103:0/2594863897 conn(0x7f98e4100250 msgr2=0x7f98e4102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 -- 172.21.15.103:0/2594863897 shutdown_connections 2024-09-11T00:25:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.428+0000 7f98eb000640 1 -- 172.21.15.103:0/2594863897 wait complete. 2024-09-11T00:25:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 Processor -- start 2024-09-11T00:25:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 -- start start 2024-09-11T00:25:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e41049b0 0x7f98e419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.433 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e4105b00 0x7f98e419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41064c0 0x7f98e419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4110620 con 0x7f98e41064c0 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41104a0 con 0x7f98e41049b0 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98eb000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41107a0 con 0x7f98e4105b00 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41064c0 0x7f98e419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41064c0 0x7f98e419c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38082/0 (socket says 172.21.15.103:38082) 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98e9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e4105b00 0x7f98e419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98ea600640 1 -- 172.21.15.103:0/1359971270 learned_addr learned my addr 172.21.15.103:0/1359971270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.429+0000 7f98e9c00640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e41049b0 0x7f98e419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e9c00640 1 -- 172.21.15.103:0/1359971270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e4105b00 msgr2=0x7f98e419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e9c00640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e4105b00 0x7f98e419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e9c00640 1 -- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41064c0 msgr2=0x7f98e419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e9c00640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41064c0 0x7f98e419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e9c00640 1 -- 172.21.15.103:0/1359971270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e419c8e0 con 0x7f98e41049b0 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98ea600640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41064c0 0x7f98e419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e9c00640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e41049b0 0x7f98e419ada0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009230 tx=0x7f98d8031a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e2000640 1 -- 172.21.15.103:0/1359971270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d803f030 con 0x7f98e41049b0 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e2000640 1 -- 172.21.15.103:0/1359971270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8003aa0 con 0x7f98e41049b0 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.430+0000 7f98e2000640 1 -- 172.21.15.103:0/1359971270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8035070 con 0x7f98e41049b0 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.431+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e419cb70 con 0x7f98e41049b0 2024-09-11T00:25:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.431+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41a32a0 con 0x7f98e41049b0 2024-09-11T00:25:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.433+0000 7f98ef446740 1 Processor -- start 2024-09-11T00:25:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ef446740 1 -- start start 2024-09-11T00:25:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ef446740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ef446740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ef446740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ef446740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b1bf0c6b0 con 0x561b1bf99970 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ef446740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b1befbae0 con 0x561b1bf9c2a0 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ef446740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561b1beff8f0 con 0x561b1bf9ece0 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98e2000640 1 -- 172.21.15.103:0/1359971270 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f98d80035f0 con 0x7f98e41049b0 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf9bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38084/0 (socket says 172.21.15.103:38084) 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98ea600640 1 -- 172.21.15.103:0/298932292 learned_addr learned my addr 172.21.15.103:0/298932292 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98e9200640 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.434+0000 7f98e9c00640 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98ea600640 1 -- 172.21.15.103:0/298932292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 msgr2=0x561b1bfa11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98ea600640 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98ea600640 1 -- 172.21.15.103:0/298932292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 msgr2=0x561b1bf9e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98ea600640 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98ea600640 1 -- 172.21.15.103:0/298932292 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b1bec8780 con 0x561b1bf99970 2024-09-11T00:25:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98ea600640 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf9bd60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0117b0 tx=0x7f98dc011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b99ce9b2f31b69a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98d6a00640 1 -- 172.21.15.103:0/298932292 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00fae0 con 0x561b1bf99970 2024-09-11T00:25:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98d6a00640 1 -- 172.21.15.103:0/298932292 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc00fc80 con 0x561b1bf99970 2024-09-11T00:25:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.435+0000 7f98d6a00640 1 -- 172.21.15.103:0/298932292 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc01a610 con 0x561b1bf99970 2024-09-11T00:25:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98e2000640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f98cc0b5360 0x7f98cc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 -- 172.21.15.103:0/298932292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 msgr2=0x561b1bf9bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf9bd60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0117b0 tx=0x7f98dc011c70 comp rx=0 tx=0).stop 2024-09-11T00:25:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98e2000640 1 -- 172.21.15.103:0/1359971270 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f98d81066e0 con 0x7f98e41049b0 2024-09-11T00:25:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98e9c00640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f98cc0b5360 0x7f98cc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.444 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 -- 172.21.15.103:0/298932292 shutdown_connections 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 --2- 172.21.15.103:0/298932292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf9bd60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 -- 172.21.15.103:0/298932292 >> 172.21.15.103:0/298932292 conn(0x561b1bf94ac0 msgr2=0x561b1bf95850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 -- 172.21.15.103:0/298932292 shutdown_connections 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.436+0000 7f98ef446740 1 -- 172.21.15.103:0/298932292 wait complete. 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.437+0000 7f98e9c00640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f98cc0b5360 0x7f98cc0b7820 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f98d80cc860 tx=0x7f98d8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.437+0000 7f98ef446740 1 -- start start 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.437+0000 7f98ef446740 1 Processor -- start 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.437+0000 7f98ef446740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bfa3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.437+0000 7f98ef446740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.437+0000 7f98ef446740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ef446740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561b1be33ab0 con 0x561b1bf99970 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ef446740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x561b1bea2570 con 0x561b1bf9c2a0 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ef446740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561b1bfa1cc0 con 0x561b1bf9ece0 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ef446740 10 client.0 _ll_register_callbacks cb 0x561b1beb1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bfa3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bfa3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38090/0 (socket says 172.21.15.103:38090) 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 -- 172.21.15.103:0/2266785883 learned_addr learned my addr 172.21.15.103:0/2266785883 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98e9200640 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 -- 172.21.15.103:0/2266785883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 msgr2=0x561b1bfa42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 -- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 msgr2=0x561b1bfa3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.438+0000 7f98ea600640 1 -- 172.21.15.103:0/2266785883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b1bc5d790 con 0x561b1bf99970 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98ea600640 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bfa3070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009e40 tx=0x7f98dc009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e8cc92a9fbccad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98cb400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98cb400640 1 -- 172.21.15.103:0/2266785883 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc02e060 con 0x561b1bf99970 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98cb400640 1 -- 172.21.15.103:0/2266785883 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f98dc012450 con 0x561b1bf99970 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc00fc80 con 0x561b1bf99970 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98cb400640 1 -- 172.21.15.103:0/2266785883 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f98dc035050 con 0x561b1bf99970 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98ef446740 10 client.15585 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561b1bcd6620 con 0x561b1bf99970 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.439+0000 7f98cb400640 1 -- 172.21.15.103:0/2266785883 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc00fc80 con 0x561b1bf99970 2024-09-11T00:25:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98cb400640 1 -- 172.21.15.103:0/2266785883 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f98dc00f040 con 0x561b1bf99970 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98ef446740 10 client.15585 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98ef446740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98ef446740 20 client.15585 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98ef446740 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1bfae6d0 0x561b1bfb0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98ef446740 4 client.15585 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b1bf0e600 con 0x561b1bfae6d0 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.440+0000 7f98e9200640 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1bfae6d0 0x561b1bfb0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.441+0000 7f98cb400640 10 client.15585 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.441+0000 7f98e9200640 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1bfae6d0 0x561b1bfb0ac0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aceb4ac8634dc83 server_cookie=6c6bb052f25d2873 in_seq=0 out_seq=0 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98cb400640 1 -- 172.21.15.103:0/2266785883 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b1bf0e600 con 0x561b1bfae6d0 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98cb400640 10 client.15585 handle_command_reply: tid=0 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 client.15585 shutdown 2024-09-11T00:25:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 msgr2=0x561b1bfa3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bfa3070 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009e40 tx=0x7f98dc009db0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 shutdown_connections 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1bfae6d0 0x561b1bfb0ac0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9ece0 0x561b1bfa42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98cb400640 0 client.15585 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 --2- 172.21.15.103:0/2266785883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bfa3070 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 >> 172.21.15.103:0/2266785883 conn(0x561b1bf94ac0 msgr2=0x561b1bf93bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 shutdown_connections 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 1 -- 172.21.15.103:0/2266785883 wait complete. 2024-09-11T00:25:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.442+0000 7f98ef446740 20 client.15585 trim_cache size 0 max 16384 2024-09-11T00:25:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.457+0000 7f98ef446740 1 Processor -- start 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.457+0000 7f98ef446740 1 -- start start 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ef446740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ef446740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ef446740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9ece0 0x561b1bf9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ef446740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41aff90 con 0x561b1bf99970 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ef446740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41ad7a0 con 0x561b1bf9ece0 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ef446740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41ab5f0 con 0x561b1bf9c2a0 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf943b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38094/0 (socket says 172.21.15.103:38094) 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 -- 172.21.15.103:0/2586755572 learned_addr learned my addr 172.21.15.103:0/2586755572 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98e9200640 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 -- 172.21.15.103:0/2586755572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 msgr2=0x561b1bf9f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 -- 172.21.15.103:0/2586755572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9ece0 msgr2=0x561b1bf9f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9ece0 0x561b1bf9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.458+0000 7f98ea600640 1 -- 172.21.15.103:0/2586755572 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b1bf9c0d0 con 0x561b1bf99970 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ea600640 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf943b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00f7b0 tx=0x7f98dc004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5533a83612920f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98d6000640 1 -- 172.21.15.103:0/2586755572 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc02e040 con 0x561b1bf99970 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98d6000640 1 -- 172.21.15.103:0/2586755572 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc005bd0 con 0x561b1bf99970 2024-09-11T00:25:43.462 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98d6000640 1 -- 172.21.15.103:0/2586755572 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc029500 con 0x561b1bf99970 2024-09-11T00:25:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ef446740 1 -- 172.21.15.103:0/2586755572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 msgr2=0x561b1bf943b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ef446740 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf943b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00f7b0 tx=0x7f98dc004a20 comp rx=0 tx=0).stop 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ef446740 1 -- 172.21.15.103:0/2586755572 shutdown_connections 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ef446740 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf9ece0 0x561b1bf9f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ef446740 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bf9f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ef446740 1 --2- 172.21.15.103:0/2586755572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf99970 0x561b1bf943b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.459+0000 7f98ef446740 1 -- 172.21.15.103:0/2586755572 >> 172.21.15.103:0/2586755572 conn(0x561b1bf94ac0 msgr2=0x561b1bf98dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 -- 172.21.15.103:0/2586755572 shutdown_connections 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 -- 172.21.15.103:0/2586755572 wait complete. 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 -- start start 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 Processor -- start 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf99970 0x561b1bfaa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf9ece0 0x561b1bfa9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41abc60 con 0x561b1bf9ece0 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41ad0d0 con 0x561b1bf99970 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41ac330 con 0x561b1bf9c2a0 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.460+0000 7f98ef446740 10 client.0 _ll_register_callbacks cb 0x7f98d000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf99970 0x561b1bfaa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf99970 0x561b1bfaa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:43136/0 (socket says 172.21.15.103:43136) 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 -- 172.21.15.103:0/1857152990 learned_addr learned my addr 172.21.15.103:0/1857152990 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf9ece0 0x561b1bfa9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98e9200640 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 -- 172.21.15.103:0/1857152990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 msgr2=0x561b1bfa95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 -- 172.21.15.103:0/1857152990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf99970 msgr2=0x561b1bfaa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf99970 0x561b1bfaa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 -- 172.21.15.103:0/1857152990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b1bfaa060 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98ea600640 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf9ece0 0x561b1bfa9af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f98dc029480 tx=0x7f98dc011500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e38f7826322861 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.461+0000 7f98cb400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98cb400640 1 -- 172.21.15.103:0/1857152990 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc005250 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98cb400640 1 -- 172.21.15.103:0/1857152990 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f98dc029500 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc005bd0 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98cb400640 1 -- 172.21.15.103:0/1857152990 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f98dc051050 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 10 client.15591 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561b1bfa8f40 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98cb400640 1 -- 172.21.15.103:0/1857152990 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc005bd0 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98cb400640 1 -- 172.21.15.103:0/1857152990 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7f98dc012450 con 0x561b1bf9ece0 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 10 client.15591 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 20 client.15591 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1c01bce0 0x561b1c01e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 4 client.15591 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.462+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b1bf0a0b0 con 0x561b1c01bce0 2024-09-11T00:25:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.463+0000 7f98e9200640 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1c01bce0 0x561b1c01e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.463+0000 7f98e9200640 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1c01bce0 0x561b1c01e100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5427db5bb8c6d984 server_cookie=30ccc317630620a6 in_seq=0 out_seq=0 2024-09-11T00:25:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.463+0000 7f98cb400640 10 client.15591 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.518+0000 7f98cb400640 1 -- 172.21.15.103:0/1857152990 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b1bf0a0b0 con 0x561b1c01bce0 2024-09-11T00:25:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.518+0000 7f98cb400640 10 client.15591 handle_command_reply: tid=0 2024-09-11T00:25:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.518+0000 7f98ef446740 1 client.15591 shutdown 2024-09-11T00:25:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf9ece0 msgr2=0x561b1bfa9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf9ece0 0x561b1bfa9af0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f98dc029480 tx=0x7f98dc011500 comp rx=0 tx=0).stop 2024-09-11T00:25:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 shutdown_connections 2024-09-11T00:25:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561b1bf9ece0 0x561b1bfa9af0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x561b1c01bce0 0x561b1c01e100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561b1bf9c2a0 0x561b1bfa95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 --2- 172.21.15.103:0/1857152990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x561b1bf99970 0x561b1bfaa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98cb400640 0 client.15591 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 >> 172.21.15.103:0/1857152990 conn(0x561b1bf94ac0 msgr2=0x561b1bf9f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 shutdown_connections 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 1 -- 172.21.15.103:0/1857152990 wait complete. 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.519+0000 7f98ef446740 20 client.15591 trim_cache size 0 max 16384 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:25:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f98cc0b5360 msgr2=0x7f98cc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f98cc0b5360 0x7f98cc0b7820 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f98d80cc860 tx=0x7f98d8009260 comp rx=0 tx=0).stop 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e41049b0 msgr2=0x7f98e419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e41049b0 0x7f98e419ada0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009230 tx=0x7f98d8031a50 comp rx=0 tx=0).stop 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 shutdown_connections 2024-09-11T00:25:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f98cc0b5360 0x7f98cc0b7820 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98e41064c0 0x7f98e419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98e4105b00 0x7f98e419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 --2- 172.21.15.103:0/1359971270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98e41049b0 0x7f98e419ada0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 >> 172.21.15.103:0/1359971270 conn(0x7f98e4100250 msgr2=0x7f98e4108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 shutdown_connections 2024-09-11T00:25:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:43.520+0000 7f98eb000640 1 -- 172.21.15.103:0/1359971270 wait complete. 2024-09-11T00:25:44.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:44 smithi183 ceph-mon[35934]: pgmap v478: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 440 KiB/s wr, 188 op/s 2024-09-11T00:25:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:44 smithi103 ceph-mon[30578]: pgmap v478: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 440 KiB/s wr, 188 op/s 2024-09-11T00:25:44.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:44 smithi117 ceph-mon[35525]: pgmap v478: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 440 KiB/s wr, 188 op/s 2024-09-11T00:25:44.825 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:25:45.215 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 -- 172.21.15.103:0/2232300921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c077ff0 msgr2=0x7fd41c1132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 --2- 172.21.15.103:0/2232300921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c077ff0 0x7fd41c1132a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd408009900 tx=0x7fd40802a410 comp rx=0 tx=0).stop 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 -- 172.21.15.103:0/2232300921 shutdown_connections 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 --2- 172.21.15.103:0/2232300921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd41c1137e0 0x7fd41c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 --2- 172.21.15.103:0/2232300921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c077ff0 0x7fd41c1132a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 --2- 172.21.15.103:0/2232300921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41c0776d0 0x7fd41c077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 -- 172.21.15.103:0/2232300921 >> 172.21.15.103:0/2232300921 conn(0x7fd41c100840 msgr2=0x7fd41c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:45.915 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 -- 172.21.15.103:0/2232300921 shutdown_connections 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.865+0000 7fd41b200640 1 -- 172.21.15.103:0/2232300921 wait complete. 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.866+0000 7fd41b200640 1 Processor -- start 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.866+0000 7fd41b200640 1 -- start start 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd41b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c0776d0 0x7fd41c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd41b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd41c077ff0 0x7fd41c19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd41b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41c1137e0 0x7fd41c1a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd41b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c119210 con 0x7fd41c0776d0 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd41b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c119090 con 0x7fd41c1137e0 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd41b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd41c119390 con 0x7fd41c077ff0 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd419e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c0776d0 0x7fd41c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd419e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c0776d0 0x7fd41c19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38124/0 (socket says 172.21.15.103:38124) 2024-09-11T00:25:45.916 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd419e00640 1 -- 172.21.15.103:0/294771704 learned_addr learned my addr 172.21.15.103:0/294771704 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd419400640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd41c077ff0 0x7fd41c19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd41a800640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41c1137e0 0x7fd41c1a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd419e00640 1 -- 172.21.15.103:0/294771704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd41c077ff0 msgr2=0x7fd41c19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.867+0000 7fd419e00640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd41c077ff0 0x7fd41c19fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd419e00640 1 -- 172.21.15.103:0/294771704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41c1137e0 msgr2=0x7fd41c1a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd419e00640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41c1137e0 0x7fd41c1a4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd419e00640 1 -- 172.21.15.103:0/294771704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41c1a56d0 con 0x7fd41c0776d0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd419e00640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c0776d0 0x7fd41c19f620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e9f0 tx=0x7fd40c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd412a00640 1 -- 172.21.15.103:0/294771704 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00cd80 con 0x7fd41c0776d0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd412a00640 1 -- 172.21.15.103:0/294771704 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd40c0108e0 con 0x7fd41c0776d0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd412a00640 1 -- 172.21.15.103:0/294771704 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c018980 con 0x7fd41c0776d0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41c1a59c0 con 0x7fd41c0776d0 2024-09-11T00:25:45.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.868+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41c1ac250 con 0x7fd41c0776d0 2024-09-11T00:25:45.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.870+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41c078be0 con 0x7fd41c0776d0 2024-09-11T00:25:45.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.872+0000 7fd412a00640 1 -- 172.21.15.103:0/294771704 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd40c010430 con 0x7fd41c0776d0 2024-09-11T00:25:45.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.874+0000 7fd412a00640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4000b5410 0x7fd4000b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:45.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.875+0000 7fd412a00640 1 -- 172.21.15.103:0/294771704 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd40c0e6710 con 0x7fd41c0776d0 2024-09-11T00:25:45.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.875+0000 7fd419400640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4000b5410 0x7fd4000b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:45.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.875+0000 7fd419400640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4000b5410 0x7fd4000b78d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd408005ab0 tx=0x7fd408005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:45.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:45.876+0000 7fd412a00640 1 -- 172.21.15.103:0/294771704 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd40c0acb90 con 0x7fd41c0776d0 2024-09-11T00:25:46.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.099+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd41c077ab0 con 0x7fd41c0776d0 2024-09-11T00:25:46.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.100+0000 7fd412a00640 1 -- 172.21.15.103:0/294771704 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11939 (secure 0 0 0) 0x7fd41c077ab0 con 0x7fd41c0776d0 2024-09-11T00:25:46.103 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:25:46.103 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":33,"btime":"2024-09-11T00:25:18:176296+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:25:17.681542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25550":{"gid":25550,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3202362307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3202362307},{"type":"v1","addr":"172.21.15.183:6835","nonce":3202362307}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:25:46.103 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 33 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4000b5410 msgr2=0x7fd4000b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4000b5410 0x7fd4000b78d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd408005ab0 tx=0x7fd408005a20 comp rx=0 tx=0).stop 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c0776d0 msgr2=0x7fd41c19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c0776d0 0x7fd41c19f620 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e9f0 tx=0x7fd40c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 shutdown_connections 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd4000b5410 0x7fd4000b78d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd41c1137e0 0x7fd41c1a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd41c077ff0 0x7fd41c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 --2- 172.21.15.103:0/294771704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd41c0776d0 0x7fd41c19f620 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:46.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 >> 172.21.15.103:0/294771704 conn(0x7fd41c100840 msgr2=0x7fd41c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:46.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 shutdown_connections 2024-09-11T00:25:46.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:46.103+0000 7fd41b200640 1 -- 172.21.15.103:0/294771704 wait complete. 2024-09-11T00:25:46.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:46 smithi103 ceph-mon[30578]: pgmap v479: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 447 KiB/s wr, 162 op/s 2024-09-11T00:25:46.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:46 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/294771704' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:46.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:46 smithi183 ceph-mon[35934]: pgmap v479: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 447 KiB/s wr, 162 op/s 2024-09-11T00:25:46.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:46 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/294771704' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:46 smithi117 ceph-mon[35525]: pgmap v479: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 447 KiB/s wr, 162 op/s 2024-09-11T00:25:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:46 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/294771704' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:47.113 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --set-id b20c3cfc --release --await' 2024-09-11T00:25:47.114 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --set-id b20c3cfc --release --await 2024-09-11T00:25:47.530 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 -- 172.21.15.103:0/156588313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc104fa0 msgr2=0x7fcddc105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 --2- 172.21.15.103:0/156588313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc104fa0 0x7fcddc105380 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009900 tx=0x7fcdcc02a410 comp rx=0 tx=0).stop 2024-09-11T00:25:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 -- 172.21.15.103:0/156588313 shutdown_connections 2024-09-11T00:25:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 --2- 172.21.15.103:0/156588313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc10b330 0x7fcddc111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 --2- 172.21.15.103:0/156588313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc10a9b0 0x7fcddc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 --2- 172.21.15.103:0/156588313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc104fa0 0x7fcddc105380 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 -- 172.21.15.103:0/156588313 >> 172.21.15.103:0/156588313 conn(0x7fcddc100a10 msgr2=0x7fcddc102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 -- 172.21.15.103:0/156588313 shutdown_connections 2024-09-11T00:25:48.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.119+0000 7fcde3c00640 1 -- 172.21.15.103:0/156588313 wait complete. 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.120+0000 7fcde3c00640 1 Processor -- start 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.120+0000 7fcde3c00640 1 -- start start 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde3c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc104fa0 0x7fcddc19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde3c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc10a9b0 0x7fcddc19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc10b330 0x7fcddc1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde3c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc114fb0 con 0x7fcddc10a9b0 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc114e30 con 0x7fcddc10b330 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde3c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc115130 con 0x7fcddc104fa0 2024-09-11T00:25:48.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc10a9b0 0x7fcddc19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc10a9b0 0x7fcddc19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38150/0 (socket says 172.21.15.103:38150) 2024-09-11T00:25:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcdd3e00640 1 -- 172.21.15.103:0/349416587 learned_addr learned my addr 172.21.15.103:0/349416587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde1200640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc104fa0 0x7fcddc19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.121+0000 7fcde1c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc10b330 0x7fcddc1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdd3e00640 1 -- 172.21.15.103:0/349416587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc104fa0 msgr2=0x7fcddc19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc104fa0 0x7fcddc19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdd3e00640 1 -- 172.21.15.103:0/349416587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc10b330 msgr2=0x7fcddc1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc10b330 0x7fcddc1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdd3e00640 1 -- 172.21.15.103:0/349416587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcddc1a1510 con 0x7fcddc10a9b0 2024-09-11T00:25:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcde1c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc10b330 0x7fcddc1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:48.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc10a9b0 0x7fcddc19b9a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e9f0 tx=0x7fcdc400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdda000640 1 -- 172.21.15.103:0/349416587 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc400cd80 con 0x7fcddc10a9b0 2024-09-11T00:25:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdda000640 1 -- 172.21.15.103:0/349416587 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc40108e0 con 0x7fcddc10a9b0 2024-09-11T00:25:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcdda000640 1 -- 172.21.15.103:0/349416587 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4018980 con 0x7fcddc10a9b0 2024-09-11T00:25:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcddc1a1800 con 0x7fcddc10a9b0 2024-09-11T00:25:48.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.122+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcddc1a7ec0 con 0x7fcddc10a9b0 2024-09-11T00:25:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.125+0000 7fcde67f7740 1 Processor -- start 2024-09-11T00:25:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde67f7740 1 -- start start 2024-09-11T00:25:48.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcdda000640 1 -- 172.21.15.103:0/349416587 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcdc4010430 con 0x7fcddc10a9b0 2024-09-11T00:25:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9cff0 0x5651b4f9d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fa9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fabc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde67f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651b4f14290 con 0x5651b4fa98a0 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde67f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651b4f036c0 con 0x5651b4f9cff0 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde67f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651b4f074d0 con 0x5651b4f9d910 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fabc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9cff0 0x5651b4f9d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fabc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38162/0 (socket says 172.21.15.103:38162) 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcdd3e00640 1 -- 172.21.15.103:0/2114397802 learned_addr learned my addr 172.21.15.103:0/2114397802 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.126+0000 7fcde1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fa9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.127+0000 7fcdd3e00640 1 -- 172.21.15.103:0/2114397802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 msgr2=0x5651b4fa9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.127+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/2114397802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fa9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.127+0000 7fcdd3e00640 1 -- 172.21.15.103:0/2114397802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9cff0 msgr2=0x5651b4f9d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.127+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/2114397802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9cff0 0x5651b4f9d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.127+0000 7fcdd3e00640 1 -- 172.21.15.103:0/2114397802 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b4e90460 con 0x5651b4fa98a0 2024-09-11T00:25:48.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.127+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/2114397802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fabc90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40ee4e0 tx=0x7fcdc40ee9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5294c77a8116b437 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.127+0000 7fcdda000640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcdc00b5360 0x7fcdc00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.128+0000 7fcdd2000640 1 -- 172.21.15.103:0/2114397802 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40f9070 con 0x5651b4fa98a0 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.128+0000 7fcdd2000640 1 -- 172.21.15.103:0/2114397802 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc4025330 con 0x5651b4fa98a0 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.128+0000 7fcdd2000640 1 -- 172.21.15.103:0/2114397802 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40ac470 con 0x5651b4fa98a0 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.128+0000 7fcdda000640 1 -- 172.21.15.103:0/349416587 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fcdc40e6220 con 0x7fcddc10a9b0 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.128+0000 7fcde1c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcdc00b5360 0x7fcdc00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 -- 172.21.15.103:0/2114397802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 msgr2=0x5651b4fabc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 --2- 172.21.15.103:0/2114397802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fabc90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcdc40ee4e0 tx=0x7fcdc40ee9a0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde1c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcdc00b5360 0x7fcdc00b7820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009fd0 tx=0x7fcdd4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 -- 172.21.15.103:0/2114397802 shutdown_connections 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 --2- 172.21.15.103:0/2114397802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fabc90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.134 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 --2- 172.21.15.103:0/2114397802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fa9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 --2- 172.21.15.103:0/2114397802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9cff0 0x5651b4f9d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 -- 172.21.15.103:0/2114397802 >> 172.21.15.103:0/2114397802 conn(0x5651b4f9ae20 msgr2=0x5651b4f9bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 -- 172.21.15.103:0/2114397802 shutdown_connections 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 -- 172.21.15.103:0/2114397802 wait complete. 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.129+0000 7fcde67f7740 1 -- start start 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 1 Processor -- start 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9d910 0x5651b4fa6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.135 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fa72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651b4d64bb0 con 0x5651b4fa98a0 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651b4e9e6c0 con 0x5651b4f9d910 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651b4fac550 con 0x5651b4f9cff0 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde67f7740 10 client.0 _ll_register_callbacks cb 0x5651b4eb9490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde1c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fa72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:42196/0 (socket says 172.21.15.103:42196) 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.130+0000 7fcde1200640 1 -- 172.21.15.103:0/932502676 learned_addr learned my addr 172.21.15.103:0/932502676 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9d910 0x5651b4fa6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcde1200640 1 -- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9d910 msgr2=0x5651b4fa6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcde1200640 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9d910 0x5651b4fa6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.136 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcde1200640 1 -- 172.21.15.103:0/932502676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 msgr2=0x5651b4fa72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcde1200640 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fa72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcde1200640 1 -- 172.21.15.103:0/932502676 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b4c61b00 con 0x5651b4f9cff0 2024-09-11T00:25:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcde1c00640 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fa72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcde1200640 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa6820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009230 tx=0x7fcdcc004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf91dedb05378fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.131+0000 7fcdcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:25:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.132+0000 7fcdcaa00640 1 -- 172.21.15.103:0/932502676 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc035040 con 0x5651b4f9cff0 2024-09-11T00:25:48.137 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.132+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc4025330 con 0x5651b4f9cff0 2024-09-11T00:25:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.132+0000 7fcdcaa00640 1 -- 172.21.15.103:0/932502676 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fcdcc005510 con 0x5651b4f9cff0 2024-09-11T00:25:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.132+0000 7fcdcaa00640 1 -- 172.21.15.103:0/932502676 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc04f050 con 0x5651b4f9cff0 2024-09-11T00:25:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.132+0000 7fcde67f7740 10 client.25657 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:48.138 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.132+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651b4cd8960 con 0x5651b4f9cff0 2024-09-11T00:25:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.132+0000 7fcdcaa00640 1 -- 172.21.15.103:0/932502676 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc4025330 con 0x5651b4f9cff0 2024-09-11T00:25:48.139 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcdcaa00640 1 -- 172.21.15.103:0/932502676 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fcdcc003990 con 0x5651b4f9cff0 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcde67f7740 10 client.25657 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcde67f7740 10 client.25657 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcde67f7740 20 client.25657 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcde67f7740 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b4fb93f0 0x5651b4fbb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcde67f7740 4 client.25657 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651b4f161e0 con 0x5651b4fb93f0 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.133+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b4fb93f0 0x5651b4fbb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.134+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b4fb93f0 0x5651b4fbb870 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=280f88ad076eb566 server_cookie=db21d1e8263417f9 in_seq=0 out_seq=0 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.134+0000 7fcdcaa00640 10 client.25657 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.134+0000 7fcdcaa00640 1 -- 172.21.15.103:0/932502676 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651b4f161e0 con 0x5651b4fb93f0 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.134+0000 7fcdcaa00640 10 client.25657 handle_command_reply: tid=0 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.134+0000 7fcde67f7740 1 client.25657 shutdown 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 msgr2=0x5651b4fa6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.140 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa6820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc009230 tx=0x7fcdcc004440 comp rx=0 tx=0).stop 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 shutdown_connections 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4fa98a0 0x5651b4fa72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4f9d910 0x5651b4fa6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b4fb93f0 0x5651b4fbb870 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 --2- 172.21.15.103:0/932502676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa6820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcdcaa00640 0 client.25657 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 >> 172.21.15.103:0/932502676 conn(0x5651b4f9ae20 msgr2=0x5651b4fa0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 shutdown_connections 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 1 -- 172.21.15.103:0/932502676 wait complete. 2024-09-11T00:25:48.141 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.135+0000 7fcde67f7740 20 client.25657 trim_cache size 0 max 16384 2024-09-11T00:25:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 Processor -- start 2024-09-11T00:25:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 -- start start 2024-09-11T00:25:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9d910 0x5651b4fa3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b4fa3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b46e0 con 0x5651b4f9d910 2024-09-11T00:25:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b1ef0 con 0x5651b4fa98a0 2024-09-11T00:25:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcde67f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1afd40 con 0x5651b4f9cff0 2024-09-11T00:25:48.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.153+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9d910 0x5651b4fa3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9d910 0x5651b4fa3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38178/0 (socket says 172.21.15.103:38178) 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcdd3e00640 1 -- 172.21.15.103:0/887079865 learned_addr learned my addr 172.21.15.103:0/887079865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1c00640 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b4fa3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1200640 1 -- 172.21.15.103:0/887079865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 msgr2=0x5651b4fa3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1200640 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b4fa3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1200640 1 -- 172.21.15.103:0/887079865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9d910 msgr2=0x5651b4fa3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1200640 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9d910 0x5651b4fa3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1200640 1 -- 172.21.15.103:0/887079865 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b4f9b8a0 con 0x5651b4f9cff0 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcde1200640 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa1060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005ab0 tx=0x7fcdcc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88b48e5fe948527 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.154+0000 7fcdd1600640 1 -- 172.21.15.103:0/887079865 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc000e60 con 0x5651b4f9cff0 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcdd1600640 1 -- 172.21.15.103:0/887079865 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc041070 con 0x5651b4f9cff0 2024-09-11T00:25:48.157 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcdd1600640 1 -- 172.21.15.103:0/887079865 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc035070 con 0x5651b4f9cff0 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 -- 172.21.15.103:0/887079865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 msgr2=0x5651b4fa1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa1060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005ab0 tx=0x7fcdcc02a410 comp rx=0 tx=0).stop 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 -- 172.21.15.103:0/887079865 shutdown_connections 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b4fa3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9d910 0x5651b4fa3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 --2- 172.21.15.103:0/887079865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9cff0 0x5651b4fa1060 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 -- 172.21.15.103:0/887079865 >> 172.21.15.103:0/887079865 conn(0x5651b4f9ae20 msgr2=0x5651b4f99db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 -- 172.21.15.103:0/887079865 shutdown_connections 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.155+0000 7fcde67f7740 1 -- 172.21.15.103:0/887079865 wait complete. 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 -- start start 2024-09-11T00:25:48.163 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 Processor -- start 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9cff0 0x5651b4fadf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b5025340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b03b0 con 0x5651b4f9cff0 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b1820 con 0x5651b4fa98a0 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.156+0000 7fcde67f7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcddc1b0a80 con 0x5651b4f9d910 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde67f7740 10 client.0 _ll_register_callbacks cb 0x7fcdcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9cff0 0x5651b4fadf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcdd3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9cff0 0x5651b4fadf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:38182/0 (socket says 172.21.15.103:38182) 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 -- 172.21.15.103:0/1872545068 learned_addr learned my addr 172.21.15.103:0/1872545068 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:48.164 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1c00640 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b5025340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 -- 172.21.15.103:0/1872545068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 msgr2=0x5651b4fae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 -- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 msgr2=0x5651b5025340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b5025340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 -- 172.21.15.103:0/1872545068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b50259b0 con 0x5651b4f9cff0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcde1200640 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9cff0 0x5651b4fadf70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02a930 tx=0x7fcdcc0055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3596ee89b9ed475 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.157+0000 7fcdcaa00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.158+0000 7fcdcaa00640 1 -- 172.21.15.103:0/1872545068 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc05f070 con 0x5651b4f9cff0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.158+0000 7fcdcaa00640 1 -- 172.21.15.103:0/1872545068 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fcdcc0046c0 con 0x5651b4f9cff0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.158+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdcc041070 con 0x5651b4f9cff0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.158+0000 7fcdcaa00640 1 -- 172.21.15.103:0/1872545068 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcdcc06b050 con 0x5651b4f9cff0 2024-09-11T00:25:48.165 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.158+0000 7fcde67f7740 10 client.15615 fetch_fsmap learned FSMap version 33 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.158+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5651b5026cd0 con 0x5651b4f9cff0 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.158+0000 7fcdcaa00640 1 -- 172.21.15.103:0/1872545068 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdcc041070 con 0x5651b4f9cff0 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcdcaa00640 1 -- 172.21.15.103:0/1872545068 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 33) ==== 8370+0+0 (secure 0 0 0) 0x7fcdcc0314a0 con 0x5651b4f9cff0 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcde67f7740 10 client.15615 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcde67f7740 10 client.15615 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcde67f7740 20 client.15615 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcde67f7740 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b5027370 0x5651b5029790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcde67f7740 4 client.15615 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b20c3cfc", "release": true, "await": true}] 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b20c3cfc", "release": true, "await": true}) -- 0x5651b4f11c90 con 0x5651b5027370 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b5027370 0x5651b5029790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcdd3e00640 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b5027370 0x5651b5029790 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a28943a2394c26bd server_cookie=bf10d402575d6166 in_seq=0 out_seq=0 2024-09-11T00:25:48.166 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.159+0000 7fcdcaa00640 10 client.15615 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:48.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcdcaa00640 1 -- 172.21.15.103:0/1872545068 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5651b4f11c90 con 0x5651b5027370 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcdcaa00640 10 client.15615 handle_command_reply: tid=0 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 client.15615 shutdown 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9cff0 msgr2=0x5651b4fadf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9cff0 0x5651b4fadf70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc02a930 tx=0x7fcdcc0055c0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 shutdown_connections 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651b4fa98a0 0x5651b5025340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651b5027370 0x5651b5029790 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651b4f9d910 0x5651b4fae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 --2- 172.21.15.103:0/1872545068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651b4f9cff0 0x5651b4fadf70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 >> 172.21.15.103:0/1872545068 conn(0x5651b4f9ae20 msgr2=0x5651b4fa50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcdcaa00640 0 client.15615 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.246+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 shutdown_connections 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.247+0000 7fcde67f7740 1 -- 172.21.15.103:0/1872545068 wait complete. 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.247+0000 7fcde67f7740 20 client.15615 trim_cache size 0 max 16384 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 33, 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 8, 2024-09-11T00:25:48.250 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "b20c3cfc": { 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 8, 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.5, 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:25:48.251 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:25:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:48.252 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:48.252 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.247+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcdc00b5360 msgr2=0x7fcdc00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.247+0000 7fcde3c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcdc00b5360 0x7fcdc00b7820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009fd0 tx=0x7fcdd4009500 comp rx=0 tx=0).stop 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.247+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc10a9b0 msgr2=0x7fcddc19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.247+0000 7fcde3c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc10a9b0 0x7fcddc19b9a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400e9f0 tx=0x7fcdc400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 shutdown_connections 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcdc00b5360 0x7fcdc00b7820 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcddc10b330 0x7fcddc1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcddc10a9b0 0x7fcddc19b9a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 --2- 172.21.15.103:0/349416587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcddc104fa0 0x7fcddc19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 >> 172.21.15.103:0/349416587 conn(0x7fcddc100a10 msgr2=0x7fcddc1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:48.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 shutdown_connections 2024-09-11T00:25:48.254 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:48.248+0000 7fcde3c00640 1 -- 172.21.15.103:0/349416587 wait complete. 2024-09-11T00:25:48.254 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:48 smithi103 ceph-mon[30578]: pgmap v480: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 358 KiB/s wr, 85 op/s 2024-09-11T00:25:48.278 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct mount_attr... yes 2024-09-11T00:25:48.402 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_qgroup_limit... yes 2024-09-11T00:25:48.504 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-09-11T00:25:48.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:48 smithi183 ceph-mon[35934]: pgmap v480: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 358 KiB/s wr, 85 op/s 2024-09-11T00:25:48.609 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-09-11T00:25:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:48 smithi117 ceph-mon[35525]: pgmap v480: 129 pgs: 129 active+clean; 47 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 358 KiB/s wr, 85 op/s 2024-09-11T00:25:48.713 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-09-11T00:25:48.813 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-09-11T00:25:48.917 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-09-11T00:25:49.021 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-09-11T00:25:49.098 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/btrfs.h usability... yes 2024-09-11T00:25:49.126 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/btrfs.h presence... yes 2024-09-11T00:25:49.126 INFO:tasks.workunit.client.0.smithi103.stdout:checking for linux/btrfs.h... yes 2024-09-11T00:25:49.234 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/btrfs_tree.h usability... yes 2024-09-11T00:25:49.286 INFO:tasks.workunit.client.0.smithi103.stdout:checking linux/btrfs_tree.h presence... yes 2024-09-11T00:25:49.286 INFO:tasks.workunit.client.0.smithi103.stdout:checking for linux/btrfs_tree.h... yes 2024-09-11T00:25:49.368 INFO:tasks.workunit.client.0.smithi103.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-09-11T00:25:49.426 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b20c3cfc', total seconds elapsed: 25.1 2024-09-11T00:25:49.426 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-11T00:25:49.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:25:49.431 INFO:tasks.workunit.client.0.smithi103.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-09-11T00:25:49.463 INFO:tasks.workunit.client.0.smithi103.stdout:configure: creating ./config.status 2024-09-11T00:25:49.822 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:50.349 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating include/builddefs 2024-09-11T00:25:50.518 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:50 smithi103 ceph-mon[30578]: pgmap v481: 129 pgs: 129 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 318 KiB/s wr, 76 op/s 2024-09-11T00:25:50.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.516+0000 7f31dae00640 1 -- 172.21.15.103:0/2177959606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 msgr2=0x7f31d410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.516+0000 7f31dae00640 1 --2- 172.21.15.103:0/2177959606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 0x7f31d410f930 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009ca0 tx=0x7f31cc02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.516+0000 7f31dae00640 1 -- 172.21.15.103:0/2177959606 shutdown_connections 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.516+0000 7f31dae00640 1 --2- 172.21.15.103:0/2177959606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 0x7f31d410f930 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.516+0000 7f31dae00640 1 --2- 172.21.15.103:0/2177959606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31d4105570 0x7f31d41059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.516+0000 7f31dae00640 1 --2- 172.21.15.103:0/2177959606 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4101220 0x7f31d4101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.516+0000 7f31dae00640 1 -- 172.21.15.103:0/2177959606 >> 172.21.15.103:0/2177959606 conn(0x7f31d4078100 msgr2=0x7f31d40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.517+0000 7f31dae00640 1 -- 172.21.15.103:0/2177959606 shutdown_connections 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.517+0000 7f31dae00640 1 -- 172.21.15.103:0/2177959606 wait complete. 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 Processor -- start 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 -- start start 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31d4101220 0x7f31d4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4105570 0x7f31d4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 0x7f31d419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31d4112df0 con 0x7f31d4105ef0 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f31d4112c70 con 0x7f31d4105570 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.518+0000 7f31dae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31d4112f70 con 0x7f31d4101220 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4105570 0x7f31d4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4105570 0x7f31d4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:43220/0 (socket says 172.21.15.103:43220) 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 -- 172.21.15.103:0/613677150 learned_addr learned my addr 172.21.15.103:0/613677150 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d8e00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 0x7f31d419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3e00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31d4101220 0x7f31d4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 -- 172.21.15.103:0/613677150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31d4101220 msgr2=0x7f31d4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31d4101220 0x7f31d4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 -- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 msgr2=0x7f31d419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 0x7f31d419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 -- 172.21.15.103:0/613677150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d419f2d0 con 0x7f31d4105570 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3e00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31d4101220 0x7f31d4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:50.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.519+0000 7f31d3400640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4105570 0x7f31d4199760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31c000ea60 tx=0x7f31c000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.520+0000 7f31d0c00640 1 -- 172.21.15.103:0/613677150 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c000cde0 con 0x7f31d4105570 2024-09-11T00:25:50.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.520+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d419f5c0 con 0x7f31d4105570 2024-09-11T00:25:50.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.520+0000 7f31d0c00640 1 -- 172.21.15.103:0/613677150 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c00035f0 con 0x7f31d4105570 2024-09-11T00:25:50.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.520+0000 7f31d0c00640 1 -- 172.21.15.103:0/613677150 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0010690 con 0x7f31d4105570 2024-09-11T00:25:50.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.520+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d41a5e50 con 0x7f31d4105570 2024-09-11T00:25:50.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.521+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d4101600 con 0x7f31d4105570 2024-09-11T00:25:50.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.524+0000 7f31d0c00640 1 -- 172.21.15.103:0/613677150 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f31c0004000 con 0x7f31d4105570 2024-09-11T00:25:50.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.525+0000 7f31d0c00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f31b80b5360 0x7f31b80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:50.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.525+0000 7f31d0c00640 1 -- 172.21.15.103:0/613677150 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f31c00e64b0 con 0x7f31d4105570 2024-09-11T00:25:50.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.526+0000 7f31d3e00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f31b80b5360 0x7f31b80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:50.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.526+0000 7f31d0c00640 1 -- 172.21.15.103:0/613677150 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f31c0056220 con 0x7f31d4105570 2024-09-11T00:25:50.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.526+0000 7f31d3e00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f31b80b5360 0x7f31b80b7820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009c20 tx=0x7f31c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:50.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:50 smithi183 ceph-mon[35934]: pgmap v481: 129 pgs: 129 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 318 KiB/s wr, 76 op/s 2024-09-11T00:25:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:50 smithi117 ceph-mon[35525]: pgmap v481: 129 pgs: 129 active+clean; 48 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 318 KiB/s wr, 76 op/s 2024-09-11T00:25:50.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.745+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31d41a6020 con 0x7f31d4105570 2024-09-11T00:25:50.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.746+0000 7f31d0c00640 1 -- 172.21.15.103:0/613677150 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11939 (secure 0 0 0) 0x7f31c00acf80 con 0x7f31d4105570 2024-09-11T00:25:50.748 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:25:50.748 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":33,"btime":"2024-09-11T00:25:18:176296+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":33,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:25:17.681542+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25550":{"gid":25550,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3202362307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3202362307},{"type":"v1","addr":"172.21.15.183:6835","nonce":3202362307}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:25:50.748 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 33 2024-09-11T00:25:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f31b80b5360 msgr2=0x7f31b80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f31b80b5360 0x7f31b80b7820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009c20 tx=0x7f31c4005f90 comp rx=0 tx=0).stop 2024-09-11T00:25:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4105570 msgr2=0x7f31d4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4105570 0x7f31d4199760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f31c000ea60 tx=0x7f31c000ef20 comp rx=0 tx=0).stop 2024-09-11T00:25:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 shutdown_connections 2024-09-11T00:25:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f31b80b5360 0x7f31b80b7820 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31d4105ef0 0x7f31d419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31d4105570 0x7f31d4199760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 --2- 172.21.15.103:0/613677150 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31d4101220 0x7f31d4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:50.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 >> 172.21.15.103:0/613677150 conn(0x7f31d4078100 msgr2=0x7f31d4106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:50.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 shutdown_connections 2024-09-11T00:25:50.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:50.748+0000 7f31dae00640 1 -- 172.21.15.103:0/613677150 wait complete. 2024-09-11T00:25:51.442 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:51 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/613677150' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:51.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:51 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/613677150' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:51 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/613677150' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:25:52.291 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:52 smithi103 ceph-mon[30578]: pgmap v482: 129 pgs: 129 active+clean; 48 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 212 KiB/s wr, 50 op/s 2024-09-11T00:25:52.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:52 smithi183 ceph-mon[35934]: pgmap v482: 129 pgs: 129 active+clean; 48 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 212 KiB/s wr, 50 op/s 2024-09-11T00:25:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:52 smithi117 ceph-mon[35525]: pgmap v482: 129 pgs: 129 active+clean; 48 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 212 KiB/s wr, 50 op/s 2024-09-11T00:25:52.732 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-11T00:25:52.732 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-11T00:25:53.147 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:25:54.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:54 smithi183 ceph-mon[35934]: pgmap v483: 129 pgs: 129 active+clean; 48 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 102 KiB/s wr, 41 op/s 2024-09-11T00:25:54.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:54 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:54.534 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:54 smithi103 ceph-mon[30578]: pgmap v483: 129 pgs: 129 active+clean; 48 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 102 KiB/s wr, 41 op/s 2024-09-11T00:25:54.534 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:54 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- 172.21.15.103:0/4262748056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b0075e00 msgr2=0x7f74b010ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 --2- 172.21.15.103:0/4262748056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b0075e00 0x7f74b010ecc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f749c009900 tx=0x7f749c02a410 comp rx=0 tx=0).stop 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- 172.21.15.103:0/4262748056 shutdown_connections 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 --2- 172.21.15.103:0/4262748056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b010f200 0x7f74b01115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 --2- 172.21.15.103:0/4262748056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b0075e00 0x7f74b010ecc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 --2- 172.21.15.103:0/4262748056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b00754e0 0x7f74b00758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- 172.21.15.103:0/4262748056 >> 172.21.15.103:0/4262748056 conn(0x7f74b00fe660 msgr2=0x7f74b0100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- 172.21.15.103:0/4262748056 shutdown_connections 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- 172.21.15.103:0/4262748056 wait complete. 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 Processor -- start 2024-09-11T00:25:54.595 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- start start 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b00754e0 0x7f74b019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b0075e00 0x7f74b019b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b010f200 0x7f74b01a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74b0114e30 con 0x7f74b00754e0 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74b0114cb0 con 0x7f74b0075e00 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74b0114fb0 con 0x7f74b010f200 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b00754e0 0x7f74b019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b00754e0 0x7f74b019b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56326/0 (socket says 172.21.15.103:56326) 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 -- 172.21.15.103:0/2101254777 learned_addr learned my addr 172.21.15.103:0/2101254777 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74afe00640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b0075e00 0x7f74b019b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5c00640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b010f200 0x7f74b01a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.596 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 -- 172.21.15.103:0/2101254777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b010f200 msgr2=0x7f74b01a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b010f200 0x7f74b01a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 -- 172.21.15.103:0/2101254777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b0075e00 msgr2=0x7f74b019b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b0075e00 0x7f74b019b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 -- 172.21.15.103:0/2101254777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b01a1350 con 0x7f74b00754e0 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b5200640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b00754e0 0x7f74b019b2a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9f0 tx=0x7f74a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74ad600640 1 -- 172.21.15.103:0/2101254777 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a400cd80 con 0x7f74b00754e0 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74ad600640 1 -- 172.21.15.103:0/2101254777 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a40054d0 con 0x7f74b00754e0 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b01a1640 con 0x7f74b00754e0 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74ad600640 1 -- 172.21.15.103:0/2101254777 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4010690 con 0x7f74b00754e0 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.462+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b01a7d00 con 0x7f74b00754e0 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 Processor -- start 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- start start 2024-09-11T00:25:54.597 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ad600640 1 -- 172.21.15.103:0/2101254777 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f74a4004e90 con 0x7f74b00754e0 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2992f0 0x563f6a2996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f6a20eca0 con 0x563f6a2a5ba0 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563f6a1fe0d0 con 0x563f6a2992f0 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563f6a201ee0 con 0x563f6a299c10 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74afe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2992f0 0x563f6a2996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74afe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56338/0 (socket says 172.21.15.103:56338) 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74afe00640 1 -- 172.21.15.103:0/3292511689 learned_addr learned my addr 172.21.15.103:0/3292511689 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5200640 1 -- 172.21.15.103:0/3292511689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2992f0 msgr2=0x563f6a2996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.598 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5200640 1 --2- 172.21.15.103:0/3292511689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2992f0 0x563f6a2996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5200640 1 -- 172.21.15.103:0/3292511689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2a5ba0 msgr2=0x563f6a2a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5200640 1 --2- 172.21.15.103:0/3292511689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5200640 1 -- 172.21.15.103:0/3292511689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6a1b0400 con 0x563f6a299c10 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5200640 1 --2- 172.21.15.103:0/3292511689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a5660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74a40e3a00 tx=0x7f74a40ee3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd1aee8391d99f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ad600640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f74980b5410 0x7f74980b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74a2000640 1 -- 172.21.15.103:0/3292511689 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a40f9070 con 0x563f6a299c10 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74a2000640 1 -- 172.21.15.103:0/3292511689 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a407b230 con 0x563f6a299c10 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5c00640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f74980b5410 0x7f74980b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ad600640 1 -- 172.21.15.103:0/2101254777 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f74a40e6350 con 0x7f74b00754e0 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74a2000640 1 -- 172.21.15.103:0/3292511689 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a40a85c0 con 0x563f6a299c10 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- 172.21.15.103:0/3292511689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 msgr2=0x563f6a2a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74b5c00640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f74980b5410 0x7f74980b78d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f74b00764c0 tx=0x7f74a80096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:54.599 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/3292511689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a5660 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74a40e3a00 tx=0x7f74a40ee3d0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- 172.21.15.103:0/3292511689 shutdown_connections 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/3292511689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/3292511689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a5660 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/3292511689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2992f0 0x563f6a2996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- 172.21.15.103:0/3292511689 >> 172.21.15.103:0/3292511689 conn(0x563f6a297120 msgr2=0x563f6a2982c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- 172.21.15.103:0/3292511689 shutdown_connections 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.467+0000 7f74ba6b9740 1 -- 172.21.15.103:0/3292511689 wait complete. 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 -- start start 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 Processor -- start 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2a5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563f6a05f7a0 con 0x563f6a2992f0 2024-09-11T00:25:54.600 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563f6a1a3a10 con 0x563f6a2a5ba0 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.472+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563f6a2a8870 con 0x563f6a299c10 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 10 client.0 _ll_register_callbacks cb 0x563f6a1b3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74afe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2a5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56342/0 (socket says 172.21.15.103:56342) 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 -- 172.21.15.103:0/185859126 learned_addr learned my addr 172.21.15.103:0/185859126 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5c00640 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 -- 172.21.15.103:0/185859126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 msgr2=0x563f6a2a27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 -- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 msgr2=0x563f6a2a2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 -- 172.21.15.103:0/185859126 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f69f6bb90 con 0x563f6a2992f0 2024-09-11T00:25:54.601 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74afe00640 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74b5200640 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2a5370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9c0 tx=0x7f74a400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b973125ffdad3b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 1 -- 172.21.15.103:0/185859126 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a4001d20 con 0x563f6a2992f0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 1 -- 172.21.15.103:0/185859126 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f74a40046c0 con 0x563f6a2992f0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74a407b230 con 0x563f6a2992f0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 1 -- 172.21.15.103:0/185859126 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f74a400d340 con 0x563f6a2992f0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 10 client.15633 fetch_fsmap learned FSMap version 34 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563f6a2b40b0 con 0x563f6a2992f0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 1 -- 172.21.15.103:0/185859126 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a407b230 con 0x563f6a2992f0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 1 -- 172.21.15.103:0/185859126 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f74a40ac980 con 0x563f6a2992f0 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 10 client.15633 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 10 client.15633 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:25:54.602 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 20 client.15633 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a2b5400 0x563f6a2b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 4 client.15633 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f6a210bf0 con 0x563f6a2b5400 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74afe00640 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a2b5400 0x563f6a2b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74afe00640 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a2b5400 0x563f6a2b77f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada31e86a9c865ad server_cookie=d4bc629364e963d7 in_seq=0 out_seq=0 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 10 client.15633 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 1 -- 172.21.15.103:0/185859126 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f6a210bf0 con 0x563f6a2b5400 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 10 client.15633 handle_command_reply: tid=0 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 client.15633 shutdown 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 msgr2=0x563f6a2a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2a5370 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9c0 tx=0x7f74a400cf20 comp rx=0 tx=0).stop 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 shutdown_connections 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a2a2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.603 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a2b5400 0x563f6a2b77f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a2a27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/185859126 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2a5370 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 >> 172.21.15.103:0/185859126 conn(0x563f6a297120 msgr2=0x563f6a29cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f7496a00640 0 client.15633 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 shutdown_connections 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 1 -- 172.21.15.103:0/185859126 wait complete. 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.473+0000 7f74ba6b9740 20 client.15633 trim_cache size 0 max 16384 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 Processor -- start 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 -- start start 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a298880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a29f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a29ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01b4460 con 0x563f6a2992f0 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01b1c70 con 0x563f6a2a5ba0 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01afac0 con 0x563f6a299c10 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a298880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a298880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56344/0 (socket says 172.21.15.103:56344) 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74b5200640 1 -- 172.21.15.103:0/4211748213 learned_addr learned my addr 172.21.15.103:0/4211748213 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74afe00640 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a29f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.491+0000 7f74b5c00640 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a29ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 -- 172.21.15.103:0/4211748213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 msgr2=0x563f6a29f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a29f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 -- 172.21.15.103:0/4211748213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 msgr2=0x563f6a29ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a29ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 -- 172.21.15.103:0/4211748213 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f6a292760 con 0x563f6a2992f0 2024-09-11T00:25:54.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5c00640 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a29ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a298880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c6a0 tx=0x7f74a400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e682a07d79e65b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74a1600640 1 -- 172.21.15.103:0/4211748213 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a400d510 con 0x563f6a2992f0 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74a1600640 1 -- 172.21.15.103:0/4211748213 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a40ac230 con 0x563f6a2992f0 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74a1600640 1 -- 172.21.15.103:0/4211748213 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a40051b0 con 0x563f6a2992f0 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/4211748213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 msgr2=0x563f6a298880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a298880 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f74a400c6a0 tx=0x7f74a400cf90 comp rx=0 tx=0).stop 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/4211748213 shutdown_connections 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a2a5ba0 0x563f6a29ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a299c10 0x563f6a29f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/4211748213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a298880 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/4211748213 >> 172.21.15.103:0/4211748213 conn(0x563f6a297120 msgr2=0x563f6a298170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/4211748213 shutdown_connections 2024-09-11T00:25:54.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/4211748213 wait complete. 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- start start 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 Processor -- start 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a299c10 0x563f6a2b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a2a5ba0 0x563f6a320f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01b0130 con 0x563f6a2992f0 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01b15a0 con 0x563f6a299c10 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74b01b0800 con 0x563f6a2a5ba0 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 10 client.0 _ll_register_callbacks cb 0x7f74a40f7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56352/0 (socket says 172.21.15.103:56352) 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 -- 172.21.15.103:0/331014418 learned_addr learned my addr 172.21.15.103:0/331014418 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:54.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5c00640 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a2a5ba0 0x563f6a320f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74afe00640 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a299c10 0x563f6a2b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 -- 172.21.15.103:0/331014418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a2a5ba0 msgr2=0x563f6a320f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a2a5ba0 0x563f6a320f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 -- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a299c10 msgr2=0x563f6a2b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a299c10 0x563f6a2b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 -- 172.21.15.103:0/331014418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f6a3214b0 con 0x563f6a2992f0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74b5200640 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2b8b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f74a40a9990 tx=0x7f74a400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc47df8c742ef68e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f7496a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f7496a00640 1 -- 172.21.15.103:0/331014418 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a40abe90 con 0x563f6a2992f0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f7496a00640 1 -- 172.21.15.103:0/331014418 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f74a40b5820 con 0x563f6a2992f0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74a40ac230 con 0x563f6a2992f0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f7496a00640 1 -- 172.21.15.103:0/331014418 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f74a40ac070 con 0x563f6a2992f0 2024-09-11T00:25:54.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 10 client.15639 fetch_fsmap learned FSMap version 34 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563f6a322630 con 0x563f6a2992f0 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f7496a00640 1 -- 172.21.15.103:0/331014418 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a40ac230 con 0x563f6a2992f0 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f7496a00640 1 -- 172.21.15.103:0/331014418 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f74a40a8070 con 0x563f6a2992f0 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 10 client.15639 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 10 client.15639 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 20 client.15639 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a322c00 0x563f6a325020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 4 client.15639 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x563f6a20c6a0 con 0x563f6a322c00 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.496+0000 7f74afe00640 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a322c00 0x563f6a325020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.501+0000 7f74afe00640 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a322c00 0x563f6a325020 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb7680b3b435d94 server_cookie=31b017d17917cd53 in_seq=0 out_seq=0 2024-09-11T00:25:54.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:54.501+0000 7f7496a00640 10 client.15639 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:54 smithi117 ceph-mon[35525]: pgmap v483: 129 pgs: 129 active+clean; 48 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 102 KiB/s wr, 41 op/s 2024-09-11T00:25:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:54 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:25:55.171 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: creating include/config.h 2024-09-11T00:25:55.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f7496a00640 1 -- 172.21.15.103:0/331014418 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563f6a20c6a0 con 0x563f6a322c00 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f7496a00640 10 client.15639 handle_command_reply: tid=0 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 client.15639 shutdown 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 msgr2=0x563f6a2b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2b8b40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f74a40a9990 tx=0x7f74a400cf20 comp rx=0 tx=0).stop 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 shutdown_connections 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563f6a322c00 0x563f6a325020 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563f6a2a5ba0 0x563f6a320f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563f6a299c10 0x563f6a2b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 --2- 172.21.15.103:0/331014418 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563f6a2992f0 0x563f6a2b8b40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f7496a00640 0 client.15639 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:55.224 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.221+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 >> 172.21.15.103:0/331014418 conn(0x563f6a297120 msgr2=0x563f6a2a3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.222+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 shutdown_connections 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.222+0000 7f74ba6b9740 1 -- 172.21.15.103:0/331014418 wait complete. 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.222+0000 7f74ba6b9740 20 client.15639 trim_cache size 0 max 16384 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 34, 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 10, 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:25:55.225 INFO:teuthology.orchestra.run.smithi103.stdout: "9f1734c5": { 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 10, 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.5, 2024-09-11T00:25:55.226 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:55.227 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:25:55.228 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:25:55.228 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.222+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f74980b5410 msgr2=0x7f74980b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.222+0000 7f74b6600640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f74980b5410 0x7f74980b78d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f74b00764c0 tx=0x7f74a80096c0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.222+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b00754e0 msgr2=0x7f74b019b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.222+0000 7f74b6600640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b00754e0 0x7f74b019b2a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f74a400e9f0 tx=0x7f74a400edf0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 shutdown_connections 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f74980b5410 0x7f74980b78d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74b010f200 0x7f74b01a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74b0075e00 0x7f74b019b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 --2- 172.21.15.103:0/2101254777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74b00754e0 0x7f74b019b2a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 >> 172.21.15.103:0/2101254777 conn(0x7f74b00fe660 msgr2=0x7f74b0111350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 shutdown_connections 2024-09-11T00:25:55.229 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:25:55.223+0000 7f74b6600640 1 -- 172.21.15.103:0/2101254777 wait complete. 2024-09-11T00:25:56.179 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9f1734c5', quiesce took 6.8 seconds. Will release after: 13.7 2024-09-11T00:25:56.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:56 smithi103 ceph-mon[30578]: pgmap v484: 129 pgs: 129 active+clean; 51 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 456 KiB/s wr, 119 op/s 2024-09-11T00:25:56.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:56 smithi183 ceph-mon[35934]: pgmap v484: 129 pgs: 129 active+clean; 51 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 456 KiB/s wr, 119 op/s 2024-09-11T00:25:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 -- 172.21.15.103:0/1558708765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa130103ae0 msgr2=0x7fa130103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 --2- 172.21.15.103:0/1558708765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa130103ae0 0x7fa130103f60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009930 tx=0x7fa11c02a420 comp rx=0 tx=0).stop 2024-09-11T00:25:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 -- 172.21.15.103:0/1558708765 shutdown_connections 2024-09-11T00:25:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 --2- 172.21.15.103:0/1558708765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1301044a0 0x7fa13010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 --2- 172.21.15.103:0/1558708765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa130103ae0 0x7fa130103f60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 --2- 172.21.15.103:0/1558708765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa130102990 0x7fa130102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 -- 172.21.15.103:0/1558708765 >> 172.21.15.103:0/1558708765 conn(0x7fa1300fe250 msgr2=0x7fa130100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 -- 172.21.15.103:0/1558708765 shutdown_connections 2024-09-11T00:25:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.636+0000 7fa136800640 1 -- 172.21.15.103:0/1558708765 wait complete. 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 Processor -- start 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 -- start start 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa130102990 0x7fa130076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130103ae0 0x7fa130075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1301044a0 0x7fa130075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa13010e650 con 0x7fa1301044a0 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa13010e4d0 con 0x7fa130103ae0 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa136800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa13010e7d0 con 0x7fa130102990 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.637+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1301044a0 0x7fa130075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1301044a0 0x7fa130075a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56374/0 (socket says 172.21.15.103:56374) 2024-09-11T00:25:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 -- 172.21.15.103:0/2813779824 learned_addr learned my addr 172.21.15.103:0/2813779824 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12ea00640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130103ae0 0x7fa130075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12f400640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa130102990 0x7fa130076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 -- 172.21.15.103:0/2813779824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa130102990 msgr2=0x7fa130076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa130102990 0x7fa130076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 -- 172.21.15.103:0/2813779824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130103ae0 msgr2=0x7fa130075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130103ae0 0x7fa130075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 -- 172.21.15.103:0/2813779824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1300762f0 con 0x7fa1301044a0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa12fe00640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1301044a0 0x7fa130075a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa12800eb10 tx=0x7fa12800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa127e00640 1 -- 172.21.15.103:0/2813779824 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128004b20 con 0x7fa1301044a0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa127e00640 1 -- 172.21.15.103:0/2813779824 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa128004cc0 con 0x7fa1301044a0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1301a2f20 con 0x7fa1301044a0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.638+0000 7fa127e00640 1 -- 172.21.15.103:0/2813779824 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128010670 con 0x7fa1301044a0 2024-09-11T00:25:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.639+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1301a33e0 con 0x7fa1301044a0 2024-09-11T00:25:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 Processor -- start 2024-09-11T00:25:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa127e00640 1 -- 172.21.15.103:0/2813779824 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa128005020 con 0x7fa1301044a0 2024-09-11T00:25:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 -- start start 2024-09-11T00:25:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc22bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc22e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22eb00 0x55bcbc230fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcbc19c6b0 con 0x55bcbc22eb00 2024-09-11T00:25:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bcbc18bae0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.642+0000 7fa13984a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bcbc18f8f0 con 0x55bcbc229790 2024-09-11T00:25:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc22e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22eb00 0x55bcbc230fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22eb00 0x55bcbc230fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56386/0 (socket says 172.21.15.103:56386) 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc22e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:49830/0 (socket says 172.21.15.103:49830) 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12ea00640 1 -- 172.21.15.103:0/4186223138 learned_addr learned my addr 172.21.15.103:0/4186223138 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12ea00640 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc22bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 -- 172.21.15.103:0/4186223138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 msgr2=0x55bcbc22bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc22bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 -- 172.21.15.103:0/4186223138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22eb00 msgr2=0x55bcbc230fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22eb00 0x55bcbc230fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 -- 172.21.15.103:0/4186223138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbc158780 con 0x55bcbc22c0c0 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12ea00640 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc22bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.643+0000 7fa12f400640 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc22e4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa120009c00 tx=0x7fa120011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fbee2167ae098df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:56.647 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa127e00640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1140b5360 0x7fa1140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa124c00640 1 -- 172.21.15.103:0/4186223138 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120004230 con 0x55bcbc22c0c0 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa127e00640 1 -- 172.21.15.103:0/2813779824 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa1280e6500 con 0x7fa1301044a0 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa124c00640 1 -- 172.21.15.103:0/4186223138 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1200043d0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa124c00640 1 -- 172.21.15.103:0/4186223138 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120004c10 con 0x55bcbc22c0c0 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa12ea00640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1140b5360 0x7fa1140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa13984a740 1 -- 172.21.15.103:0/4186223138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 msgr2=0x55bcbc22e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.644+0000 7fa12ea00640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1140b5360 0x7fa1140b7820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc22e4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa120009c00 tx=0x7fa120011810 comp rx=0 tx=0).stop 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 -- 172.21.15.103:0/4186223138 shutdown_connections 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22eb00 0x55bcbc230fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc22e4b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 --2- 172.21.15.103:0/4186223138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc22bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.648 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 -- 172.21.15.103:0/4186223138 >> 172.21.15.103:0/4186223138 conn(0x55bcbc2248e0 msgr2=0x55bcbc225670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 -- 172.21.15.103:0/4186223138 shutdown_connections 2024-09-11T00:25:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 -- 172.21.15.103:0/4186223138 wait complete. 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 -- start start 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.645+0000 7fa13984a740 1 Processor -- start 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa13984a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc229790 0x55bcbc232e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa13984a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa13984a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc2340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa13984a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcbc0c3ab0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa13984a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bcbc132570 con 0x55bcbc229790 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa13984a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bcbc231ae0 con 0x55bcbc22eb00 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa13984a740 10 client.0 _ll_register_callbacks cb 0x55bcbc141100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2334d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56398/0 (socket says 172.21.15.103:56398) 2024-09-11T00:25:56.653 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 -- 172.21.15.103:0/1459334959 learned_addr learned my addr 172.21.15.103:0/1459334959 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12f400640 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc229790 0x55bcbc232e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12fe00640 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc2340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 -- 172.21.15.103:0/1459334959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 msgr2=0x55bcbc2340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc2340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 -- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc229790 msgr2=0x55bcbc232e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc229790 0x55bcbc232e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.646+0000 7fa12ea00640 1 -- 172.21.15.103:0/1459334959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbbeed790 con 0x55bcbc22c0c0 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa12ea00640 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2334d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0318f0 tx=0x7fa11c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204aa127860e492 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa119600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa119600640 1 -- 172.21.15.103:0/1459334959 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c035040 con 0x55bcbc22c0c0 2024-09-11T00:25:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1200043d0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa119600640 1 -- 172.21.15.103:0/1459334959 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa11c003ac0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa119600640 1 -- 172.21.15.103:0/1459334959 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa11c003cc0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa13984a740 10 client.15651 fetch_fsmap learned FSMap version 34 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.647+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bcbbf66620 con 0x55bcbc22c0c0 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa119600640 1 -- 172.21.15.103:0/1459334959 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1200043d0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa119600640 1 -- 172.21.15.103:0/1459334959 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fa11c03d3f0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa13984a740 10 client.15651 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa13984a740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa13984a740 20 client.15651 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa13984a740 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc23e770 0x55bcbc240bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa13984a740 4 client.15651 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:25:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcbc19e600 con 0x55bcbc23e770 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.648+0000 7fa12f400640 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc23e770 0x55bcbc240bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.649+0000 7fa12f400640 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc23e770 0x55bcbc240bf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab7aaec759033fc server_cookie=b4a7e26e216b982 in_seq=0 out_seq=0 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.649+0000 7fa119600640 10 client.15651 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa119600640 1 -- 172.21.15.103:0/1459334959 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bcbc19e600 con 0x55bcbc23e770 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa119600640 10 client.15651 handle_command_reply: tid=0 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 client.15651 shutdown 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 msgr2=0x55bcbc2334d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2334d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0318f0 tx=0x7fa11c02a420 comp rx=0 tx=0).stop 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 shutdown_connections 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc23e770 0x55bcbc240bf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc2340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2334d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 --2- 172.21.15.103:0/1459334959 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc229790 0x55bcbc232e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 >> 172.21.15.103:0/1459334959 conn(0x55bcbc2248e0 msgr2=0x55bcbc223a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa119600640 0 client.15651 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 shutdown_connections 2024-09-11T00:25:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 1 -- 172.21.15.103:0/1459334959 wait complete. 2024-09-11T00:25:56.657 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.650+0000 7fa13984a740 20 client.15651 trim_cache size 0 max 16384 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 Processor -- start 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 -- start start 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc228b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22eb00 0x55bcbc223240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301b01b0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301ad9c0 con 0x55bcbc22eb00 2024-09-11T00:25:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.665+0000 7fa13984a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301ab810 con 0x55bcbc229790 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2290d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56408/0 (socket says 172.21.15.103:56408) 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 -- 172.21.15.103:0/3379973209 learned_addr learned my addr 172.21.15.103:0/3379973209 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12fe00640 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22eb00 0x55bcbc223240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12f400640 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc228b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 -- 172.21.15.103:0/3379973209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 msgr2=0x55bcbc228b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc228b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 -- 172.21.15.103:0/3379973209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22eb00 msgr2=0x55bcbc223240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22eb00 0x55bcbc223240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 -- 172.21.15.103:0/3379973209 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbc232930 con 0x55bcbc22c0c0 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12f400640 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc228b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:25:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.666+0000 7fa12ea00640 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2290d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009850 tx=0x7fa11c004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f9289c27cf4d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa11be00640 1 -- 172.21.15.103:0/3379973209 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c065070 con 0x55bcbc22c0c0 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa11be00640 1 -- 172.21.15.103:0/3379973209 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c009520 con 0x55bcbc22c0c0 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa11be00640 1 -- 172.21.15.103:0/3379973209 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c035040 con 0x55bcbc22c0c0 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 -- 172.21.15.103:0/3379973209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 msgr2=0x55bcbc2290d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2290d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009850 tx=0x7fa11c004400 comp rx=0 tx=0).stop 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 -- 172.21.15.103:0/3379973209 shutdown_connections 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22eb00 0x55bcbc223240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2290d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 --2- 172.21.15.103:0/3379973209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc229790 0x55bcbc228b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 -- 172.21.15.103:0/3379973209 >> 172.21.15.103:0/3379973209 conn(0x55bcbc2248e0 msgr2=0x55bcbc225380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 -- 172.21.15.103:0/3379973209 shutdown_connections 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.667+0000 7fa13984a740 1 -- 172.21.15.103:0/3379973209 wait complete. 2024-09-11T00:25:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 -- start start 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 Processor -- start 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc229790 0x55bcbc23a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2396b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc239c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301abe80 con 0x55bcbc229790 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301ad2f0 con 0x55bcbc22c0c0 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1301ac550 con 0x55bcbc22eb00 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa13984a740 10 client.0 _ll_register_callbacks cb 0x7fa11c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa12f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc229790 0x55bcbc23a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa12f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc229790 0x55bcbc23a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:56412/0 (socket says 172.21.15.103:56412) 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa12f400640 1 -- 172.21.15.103:0/2050858723 learned_addr learned my addr 172.21.15.103:0/2050858723 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa12fe00640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc239c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.668+0000 7fa12ea00640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa12f400640 1 -- 172.21.15.103:0/2050858723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 msgr2=0x55bcbc239c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa12f400640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc239c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa12f400640 1 -- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 msgr2=0x55bcbc2396b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa12f400640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2396b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa12f400640 1 -- 172.21.15.103:0/2050858723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbc23a4a0 con 0x55bcbc229790 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa12ea00640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa12f400640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc229790 0x55bcbc23a9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa120005fd0 tx=0x7fa120009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4785d076a65c35b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa119600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa119600640 1 -- 172.21.15.103:0/2050858723 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa120032060 con 0x55bcbc229790 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa119600640 1 -- 172.21.15.103:0/2050858723 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa120004450 con 0x55bcbc229790 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.669+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa11c009520 con 0x55bcbc229790 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa119600640 1 -- 172.21.15.103:0/2050858723 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa120039050 con 0x55bcbc229790 2024-09-11T00:25:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 10 client.15657 fetch_fsmap learned FSMap version 34 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bcbc2b3ff0 con 0x55bcbc229790 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa119600640 1 -- 172.21.15.103:0/2050858723 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c009520 con 0x55bcbc229790 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa119600640 1 -- 172.21.15.103:0/2050858723 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fa120024040 con 0x55bcbc229790 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 10 client.15657 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 20 client.15657 populate_metadata read hostname 'smithi103' 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc2b4890 0x55bcbc2b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 4 client.15657 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcbc19a0b0 con 0x55bcbc2b4890 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.670+0000 7fa12ea00640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc2b4890 0x55bcbc2b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:25:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.671+0000 7fa12ea00640 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc2b4890 0x55bcbc2b6cb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1ded9ee740d37e server_cookie=e5ac0bd16dcb5cc8 in_seq=0 out_seq=0 2024-09-11T00:25:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.671+0000 7fa119600640 10 client.15657 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:56.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:56 smithi117 ceph-mon[35525]: pgmap v484: 129 pgs: 129 active+clean; 51 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 456 KiB/s wr, 119 op/s 2024-09-11T00:25:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa119600640 1 -- 172.21.15.103:0/2050858723 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcbc19a0b0 con 0x55bcbc2b4890 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa119600640 10 client.15657 handle_command_reply: tid=0 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 client.15657 shutdown 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc229790 msgr2=0x55bcbc23a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc229790 0x55bcbc23a9b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa120005fd0 tx=0x7fa120009780 comp rx=0 tx=0).stop 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 shutdown_connections 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bcbc2b4890 0x55bcbc2b6cb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bcbc22eb00 0x55bcbc239c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bcbc22c0c0 0x55bcbc2396b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 --2- 172.21.15.103:0/2050858723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcbc229790 0x55bcbc23a9b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa119600640 0 client.15657 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 >> 172.21.15.103:0/2050858723 conn(0x55bcbc2248e0 msgr2=0x55bcbc227930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 shutdown_connections 2024-09-11T00:25:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.696+0000 7fa13984a740 1 -- 172.21.15.103:0/2050858723 wait complete. 2024-09-11T00:25:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa13984a740 20 client.15657 trim_cache size 0 max 16384 2024-09-11T00:25:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:25:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:25:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:25:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:25:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1140b5360 msgr2=0x7fa1140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1140b5360 0x7fa1140b7820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa11c000c00 tx=0x7fa11c005e50 comp rx=0 tx=0).stop 2024-09-11T00:25:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1301044a0 msgr2=0x7fa130075a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:25:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1301044a0 0x7fa130075a80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa12800eb10 tx=0x7fa12800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 shutdown_connections 2024-09-11T00:25:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa1140b5360 0x7fa1140b7820 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1301044a0 0x7fa130075a80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa130103ae0 0x7fa130075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 --2- 172.21.15.103:0/2813779824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa130102990 0x7fa130076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:25:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 >> 172.21.15.103:0/2813779824 conn(0x7fa1300fe250 msgr2=0x7fa130104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:25:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.697+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 shutdown_connections 2024-09-11T00:25:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:25:56.698+0000 7fa136800640 1 -- 172.21.15.103:0/2813779824 wait complete. 2024-09-11T00:25:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:25:58 smithi103 ceph-mon[30578]: pgmap v485: 129 pgs: 129 active+clean; 51 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 514 KiB/s wr, 136 op/s 2024-09-11T00:25:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:25:58 smithi117 ceph-mon[35525]: pgmap v485: 129 pgs: 129 active+clean; 51 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 514 KiB/s wr, 136 op/s 2024-09-11T00:25:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:25:58 smithi183 ceph-mon[35934]: pgmap v485: 129 pgs: 129 active+clean; 51 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 514 KiB/s wr, 136 op/s 2024-09-11T00:26:00.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:00 smithi103 ceph-mon[30578]: pgmap v486: 129 pgs: 129 active+clean; 51 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 521 KiB/s wr, 148 op/s 2024-09-11T00:26:00.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:00 smithi117 ceph-mon[35525]: pgmap v486: 129 pgs: 129 active+clean; 51 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 521 KiB/s wr, 148 op/s 2024-09-11T00:26:00.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:00 smithi183 ceph-mon[35934]: pgmap v486: 129 pgs: 129 active+clean; 51 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 521 KiB/s wr, 148 op/s 2024-09-11T00:26:02.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:02 smithi103 ceph-mon[30578]: pgmap v487: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 501 KiB/s wr, 144 op/s 2024-09-11T00:26:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:02 smithi117 ceph-mon[35525]: pgmap v487: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 501 KiB/s wr, 144 op/s 2024-09-11T00:26:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:02 smithi183 ceph-mon[35934]: pgmap v487: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 501 KiB/s wr, 144 op/s 2024-09-11T00:26:04.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:04 smithi103 ceph-mon[30578]: pgmap v488: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 448 KiB/s wr, 136 op/s 2024-09-11T00:26:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:04 smithi117 ceph-mon[35525]: pgmap v488: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 448 KiB/s wr, 136 op/s 2024-09-11T00:26:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:04 smithi183 ceph-mon[35934]: pgmap v488: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 448 KiB/s wr, 136 op/s 2024-09-11T00:26:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:06 smithi103 ceph-mon[30578]: pgmap v489: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 433 KiB/s wr, 133 op/s 2024-09-11T00:26:06.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:06 smithi117 ceph-mon[35525]: pgmap v489: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 433 KiB/s wr, 133 op/s 2024-09-11T00:26:06.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:06 smithi183 ceph-mon[35934]: pgmap v489: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 433 KiB/s wr, 133 op/s 2024-09-11T00:26:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:08 smithi103 ceph-mon[30578]: pgmap v490: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-09-11T00:26:08.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:08 smithi117 ceph-mon[35525]: pgmap v490: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-09-11T00:26:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:08 smithi183 ceph-mon[35934]: pgmap v490: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-09-11T00:26:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.821+0000 7fe139e00640 1 -- 172.21.15.103:0/101008850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe134102990 msgr2=0x7fe134102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.821+0000 7fe139e00640 1 --2- 172.21.15.103:0/101008850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe134102990 0x7fe134102d90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe124009930 tx=0x7fe12402a440 comp rx=0 tx=0).stop 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.822+0000 7fe139e00640 1 -- 172.21.15.103:0/101008850 shutdown_connections 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.822+0000 7fe139e00640 1 --2- 172.21.15.103:0/101008850 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1341044a0 0x7fe13410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.822+0000 7fe139e00640 1 --2- 172.21.15.103:0/101008850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 0x7fe134103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.822+0000 7fe139e00640 1 --2- 172.21.15.103:0/101008850 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe134102990 0x7fe134102d90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.822+0000 7fe139e00640 1 -- 172.21.15.103:0/101008850 >> 172.21.15.103:0/101008850 conn(0x7fe1340fe250 msgr2=0x7fe134100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.822+0000 7fe139e00640 1 -- 172.21.15.103:0/101008850 shutdown_connections 2024-09-11T00:26:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.822+0000 7fe139e00640 1 -- 172.21.15.103:0/101008850 wait complete. 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 Processor -- start 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 -- start start 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe134102990 0x7fe134076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 0x7fe134075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1341044a0 0x7fe134075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe13410e5e0 con 0x7fe1341044a0 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe13410e460 con 0x7fe134102990 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe139e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe13410e760 con 0x7fe134103ae0 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe133400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1341044a0 0x7fe134075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe12a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 0x7fe134075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.823+0000 7fe12a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 0x7fe134075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:49316/0 (socket says 172.21.15.103:49316) 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe12a000640 1 -- 172.21.15.103:0/1773672445 learned_addr learned my addr 172.21.15.103:0/1773672445 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe132a00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe134102990 0x7fe134076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe12a000640 1 -- 172.21.15.103:0/1773672445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe134102990 msgr2=0x7fe134076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe12a000640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe134102990 0x7fe134076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe12a000640 1 -- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1341044a0 msgr2=0x7fe134075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe12a000640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1341044a0 0x7fe134075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe12a000640 1 -- 172.21.15.103:0/1773672445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1340762f0 con 0x7fe134103ae0 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe133400640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1341044a0 0x7fe134075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe12a000640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 0x7fe134075540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00e9f0 tx=0x7fe11c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe138a00640 1 -- 172.21.15.103:0/1773672445 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00cd80 con 0x7fe134103ae0 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.824+0000 7fe138a00640 1 -- 172.21.15.103:0/1773672445 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11c0108e0 con 0x7fe134103ae0 2024-09-11T00:26:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.825+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1341a2ea0 con 0x7fe134103ae0 2024-09-11T00:26:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.825+0000 7fe138a00640 1 -- 172.21.15.103:0/1773672445 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c018980 con 0x7fe134103ae0 2024-09-11T00:26:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.825+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1341a3360 con 0x7fe134103ae0 2024-09-11T00:26:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 Processor -- start 2024-09-11T00:26:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 -- start start 2024-09-11T00:26:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe138a00640 1 -- 172.21.15.103:0/1773672445 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe11c010430 con 0x7fe134103ae0 2024-09-11T00:26:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c4252d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c42523f6b0 con 0x55c4252d1b00 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c42522eae0 con 0x55c4252cf0c0 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe13ce33740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c4252328f0 con 0x55c4252cc790 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.828+0000 7fe12a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252ceb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:49332/0 (socket says 172.21.15.103:49332) 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 -- 172.21.15.103:0/3098679172 learned_addr learned my addr 172.21.15.103:0/3098679172 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe132a00640 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c4252d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 -- 172.21.15.103:0/3098679172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 msgr2=0x55c4252d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c4252d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 -- 172.21.15.103:0/3098679172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 msgr2=0x55c4252d3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 -- 172.21.15.103:0/3098679172 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4251fb780 con 0x55c4252cc790 2024-09-11T00:26:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe133400640 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252ceb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe12c0117e0 tx=0x7fe12c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6562d3c4e11d136 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe123e00640 1 -- 172.21.15.103:0/3098679172 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c011310 con 0x55c4252cc790 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe123e00640 1 -- 172.21.15.103:0/3098679172 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe12c003e20 con 0x55c4252cc790 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.829+0000 7fe123e00640 1 -- 172.21.15.103:0/3098679172 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0044f0 con 0x55c4252cc790 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe138a00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1180b52e0 0x7fe1180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe138a00640 1 -- 172.21.15.103:0/1773672445 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe11c0e6310 con 0x7fe134103ae0 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe132a00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1180b52e0 0x7fe1180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 -- 172.21.15.103:0/3098679172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 msgr2=0x55c4252ceb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252ceb80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe12c0117e0 tx=0x7fe12c011ca0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 -- 172.21.15.103:0/3098679172 shutdown_connections 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c4252d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3098679172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252ceb80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 -- 172.21.15.103:0/3098679172 >> 172.21.15.103:0/3098679172 conn(0x55c4252c78e0 msgr2=0x55c4252c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.830+0000 7fe13ce33740 1 -- 172.21.15.103:0/3098679172 shutdown_connections 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.831+0000 7fe13ce33740 1 -- 172.21.15.103:0/3098679172 wait complete. 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.831+0000 7fe132a00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1180b52e0 0x7fe1180b77a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe124006010 tx=0x7fe1240314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.831+0000 7fe13ce33740 1 -- start start 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.831+0000 7fe13ce33740 1 Processor -- start 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cf0c0 0x55c4252d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe13ce33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c425166ab0 con 0x55c4252d1b00 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe13ce33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c4251d5570 con 0x55c4252cc790 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe13ce33740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c4252d4ae0 con 0x55c4252cf0c0 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe13ce33740 10 client.0 _ll_register_callbacks cb 0x55c4251e4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe12a000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cf0c0 0x55c4252d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57564/0 (socket says 172.21.15.103:57564) 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 -- 172.21.15.103:0/200266190 learned_addr learned my addr 172.21.15.103:0/200266190 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe132a00640 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 -- 172.21.15.103:0/200266190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cf0c0 msgr2=0x55c4252d6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cf0c0 0x55c4252d6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 -- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 msgr2=0x55c4252d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe133400640 1 -- 172.21.15.103:0/200266190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c424f90790 con 0x55c4252d1b00 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.832+0000 7fe12a000640 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cf0c0 0x55c4252d6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe133400640 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d7100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00a010 tx=0x7fe12c011340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4be4f73a1d40e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe120c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:26:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe120c00640 1 -- 172.21.15.103:0/200266190 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c02a060 con 0x55c4252d1b00 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe120c00640 1 -- 172.21.15.103:0/200266190 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe12c00f9a0 con 0x55c4252d1b00 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe12c003e20 con 0x55c4252d1b00 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe120c00640 1 -- 172.21.15.103:0/200266190 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe12c031050 con 0x55c4252d1b00 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe13ce33740 10 client.15669 fetch_fsmap learned FSMap version 34 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c425009620 con 0x55c4252d1b00 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.833+0000 7fe120c00640 1 -- 172.21.15.103:0/200266190 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe12c003e20 con 0x55c4252d1b00 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe120c00640 1 -- 172.21.15.103:0/200266190 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fe12c018020 con 0x55c4252d1b00 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe13ce33740 10 client.15669 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe13ce33740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe13ce33740 20 client.15669 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe13ce33740 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c4252e1590 0x55c4252e39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe13ce33740 4 client.15669 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c425241600 con 0x55c4252e1590 2024-09-11T00:26:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.834+0000 7fe132a00640 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c4252e1590 0x55c4252e39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.835+0000 7fe132a00640 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c4252e1590 0x55c4252e39b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e9af377637bb46 server_cookie=8b06183ace2b4a8e in_seq=0 out_seq=0 2024-09-11T00:26:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.835+0000 7fe120c00640 10 client.15669 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.835+0000 7fe120c00640 1 -- 172.21.15.103:0/200266190 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c425241600 con 0x55c4252e1590 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.835+0000 7fe120c00640 10 client.15669 handle_command_reply: tid=0 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 client.15669 shutdown 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 msgr2=0x55c4252d7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d7100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00a010 tx=0x7fe12c011340 comp rx=0 tx=0).stop 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 shutdown_connections 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c4252d7100 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c4252e1590 0x55c4252e39b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cf0c0 0x55c4252d6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 --2- 172.21.15.103:0/200266190 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe120c00640 0 client.15669 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 >> 172.21.15.103:0/200266190 conn(0x55c4252c78e0 msgr2=0x55c4252c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 shutdown_connections 2024-09-11T00:26:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 1 -- 172.21.15.103:0/200266190 wait complete. 2024-09-11T00:26:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.836+0000 7fe13ce33740 20 client.15669 trim_cache size 0 max 16384 2024-09-11T00:26:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 Processor -- start 2024-09-11T00:26:09.853 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 -- start start 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252cf0c0 0x55c4252cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252d1b00 0x55c4252e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341b0270 con 0x55c4252cf0c0 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ada80 con 0x55c4252cc790 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.851+0000 7fe13ce33740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ab8d0 con 0x55c4252d1b00 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe12a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252cf0c0 0x55c4252cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe132a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe12a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252cf0c0 0x55c4252cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57570/0 (socket says 172.21.15.103:57570) 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe12a000640 1 -- 172.21.15.103:0/3095054865 learned_addr learned my addr 172.21.15.103:0/3095054865 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe133400640 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252d1b00 0x55c4252e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe132a00640 1 -- 172.21.15.103:0/3095054865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252d1b00 msgr2=0x55c4252e82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.854 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe132a00640 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252d1b00 0x55c4252e82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe132a00640 1 -- 172.21.15.103:0/3095054865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252cf0c0 msgr2=0x55c4252cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe132a00640 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252cf0c0 0x55c4252cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe132a00640 1 -- 172.21.15.103:0/3095054865 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4252d1820 con 0x55c4252cc790 2024-09-11T00:26:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.852+0000 7fe132a00640 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252ca720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe124005ab0 tx=0x7fe124040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d704c15896539a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe123400640 1 -- 172.21.15.103:0/3095054865 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124035040 con 0x55c4252cc790 2024-09-11T00:26:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe123400640 1 -- 172.21.15.103:0/3095054865 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe124003c20 con 0x55c4252cc790 2024-09-11T00:26:09.855 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe123400640 1 -- 172.21.15.103:0/3095054865 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12403dba0 con 0x55c4252cc790 2024-09-11T00:26:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 -- 172.21.15.103:0/3095054865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 msgr2=0x55c4252ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252ca720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe124005ab0 tx=0x7fe124040000 comp rx=0 tx=0).stop 2024-09-11T00:26:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 -- 172.21.15.103:0/3095054865 shutdown_connections 2024-09-11T00:26:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252d1b00 0x55c4252e82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252cf0c0 0x55c4252cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.858 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 --2- 172.21.15.103:0/3095054865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cc790 0x55c4252ca720 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 -- 172.21.15.103:0/3095054865 >> 172.21.15.103:0/3095054865 conn(0x55c4252c78e0 msgr2=0x55c4252cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 -- 172.21.15.103:0/3095054865 shutdown_connections 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.853+0000 7fe13ce33740 1 -- 172.21.15.103:0/3095054865 wait complete. 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 -- start start 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 Processor -- start 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252dda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c42534a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c42534cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341abf40 con 0x55c4252d1b00 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ad3b0 con 0x55c4252cf0c0 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.854+0000 7fe13ce33740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1341ac610 con 0x55c4252cc790 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe13ce33740 10 client.0 _ll_register_callbacks cb 0x7fe11c0e9c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe132a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252dda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe133400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c42534cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe132a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252dda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:49352/0 (socket says 172.21.15.103:49352) 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe132a00640 1 -- 172.21.15.103:0/1865177451 learned_addr learned my addr 172.21.15.103:0/1865177451 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe12a000640 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c42534a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe133400640 1 -- 172.21.15.103:0/1865177451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 msgr2=0x55c4252dda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe133400640 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252dda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe133400640 1 -- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 msgr2=0x55c42534a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe133400640 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c42534a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe133400640 1 -- 172.21.15.103:0/1865177451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c42534d430 con 0x55c4252d1b00 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe132a00640 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe133400640 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c42534cc00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe12c006010 tx=0x7fe12c0116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a60bf3021e81db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.855+0000 7fe120c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe120c00640 1 -- 172.21.15.103:0/1865177451 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c03a060 con 0x55c4252d1b00 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe120c00640 1 -- 172.21.15.103:0/1865177451 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe12c0113b0 con 0x55c4252d1b00 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe124003c20 con 0x55c4252d1b00 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe120c00640 1 -- 172.21.15.103:0/1865177451 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe12c02a070 con 0x55c4252d1b00 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 10 client.15675 fetch_fsmap learned FSMap version 34 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c42534e740 con 0x55c4252d1b00 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe120c00640 1 -- 172.21.15.103:0/1865177451 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe124003c20 con 0x55c4252d1b00 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe120c00640 1 -- 172.21.15.103:0/1865177451 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fe12c018040 con 0x55c4252d1b00 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 10 client.15675 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 20 client.15675 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c42534f120 0x55c425351540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 4 client.15675 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.856+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c42523d0b0 con 0x55c42534f120 2024-09-11T00:26:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.857+0000 7fe132a00640 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c42534f120 0x55c425351540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.857+0000 7fe132a00640 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c42534f120 0x55c425351540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a63e103bd32de4 server_cookie=1a5bde01a1298f45 in_seq=0 out_seq=0 2024-09-11T00:26:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.857+0000 7fe120c00640 10 client.15675 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.871+0000 7fe120c00640 1 -- 172.21.15.103:0/1865177451 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c42523d0b0 con 0x55c42534f120 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.871+0000 7fe120c00640 10 client.15675 handle_command_reply: tid=0 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.871+0000 7fe13ce33740 1 client.15675 shutdown 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 msgr2=0x55c42534cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c42534cc00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe12c006010 tx=0x7fe12c0116e0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 shutdown_connections 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c4252d1b00 0x55c42534cc00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c4252cf0c0 0x55c42534a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.873 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c42534f120 0x55c425351540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 --2- 172.21.15.103:0/1865177451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c4252cc790 0x55c4252dda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe120c00640 0 client.15675 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 >> 172.21.15.103:0/1865177451 conn(0x55c4252c78e0 msgr2=0x55c4252cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 shutdown_connections 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 1 -- 172.21.15.103:0/1865177451 wait complete. 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.872+0000 7fe13ce33740 20 client.15675 trim_cache size 0 max 16384 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:26:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:26:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1180b52e0 msgr2=0x7fe1180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1180b52e0 0x7fe1180b77a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe124006010 tx=0x7fe1240314a0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 msgr2=0x7fe134075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 0x7fe134075540 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00e9f0 tx=0x7fe11c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 shutdown_connections 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe1180b52e0 0x7fe1180b77a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1341044a0 0x7fe134075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe134103ae0 0x7fe134075540 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 --2- 172.21.15.103:0/1773672445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe134102990 0x7fe134076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 >> 172.21.15.103:0/1773672445 conn(0x7fe1340fe250 msgr2=0x7fe134106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 shutdown_connections 2024-09-11T00:26:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:09.873+0000 7fe139e00640 1 -- 172.21.15.103:0/1773672445 wait complete. 2024-09-11T00:26:09.881 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:26:10.254 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:26:10.590 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:10 smithi103 ceph-mon[30578]: pgmap v491: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 KiB/s wr, 29 op/s 2024-09-11T00:26:10.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:10 smithi117 ceph-mon[35525]: pgmap v491: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 KiB/s wr, 29 op/s 2024-09-11T00:26:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:10 smithi183 ceph-mon[35934]: pgmap v491: 129 pgs: 129 active+clean; 51 MiB data, 615 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 KiB/s wr, 29 op/s 2024-09-11T00:26:10.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 -- 172.21.15.103:0/2209134362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c4101850 msgr2=0x7f97c4101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:10.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 --2- 172.21.15.103:0/2209134362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c4101850 0x7f97c4101c30 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009900 tx=0x7f97b402a3f0 comp rx=0 tx=0).stop 2024-09-11T00:26:10.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 -- 172.21.15.103:0/2209134362 shutdown_connections 2024-09-11T00:26:10.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 --2- 172.21.15.103:0/2209134362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c410f400 0x7f97c41117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:10.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 --2- 172.21.15.103:0/2209134362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97c4102170 0x7f97c410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 --2- 172.21.15.103:0/2209134362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c4101850 0x7f97c4101c30 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 -- 172.21.15.103:0/2209134362 >> 172.21.15.103:0/2209134362 conn(0x7f97c40fd700 msgr2=0x7f97c40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 -- 172.21.15.103:0/2209134362 shutdown_connections 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.886+0000 7f97c8e00640 1 -- 172.21.15.103:0/2209134362 wait complete. 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.887+0000 7f97c8e00640 1 Processor -- start 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.887+0000 7f97c8e00640 1 -- start start 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c8e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c4101850 0x7f97c419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c8e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97c4102170 0x7f97c419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c8e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c410f400 0x7f97c41a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c8e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4114fd0 con 0x7f97c410f400 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c8e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4114e50 con 0x7f97c4102170 2024-09-11T00:26:10.895 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c8e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4115150 con 0x7f97c4101850 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c410f400 0x7f97c41a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c410f400 0x7f97c41a0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57596/0 (socket says 172.21.15.103:57596) 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 -- 172.21.15.103:0/224010900 learned_addr learned my addr 172.21.15.103:0/224010900 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c1400640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97c4102170 0x7f97c419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c1e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c4101850 0x7f97c419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 -- 172.21.15.103:0/224010900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c4101850 msgr2=0x7f97c419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c4101850 0x7f97c419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 -- 172.21.15.103:0/224010900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97c4102170 msgr2=0x7f97c419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97c4102170 0x7f97c419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c2800640 1 -- 172.21.15.103:0/224010900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c41a1550 con 0x7f97c410f400 2024-09-11T00:26:10.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.888+0000 7f97c1e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c4101850 0x7f97c419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.889+0000 7f97c2800640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c410f400 0x7f97c41a0d80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00eb40 tx=0x7f97bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.889+0000 7f97baa00640 1 -- 172.21.15.103:0/224010900 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc004b20 con 0x7f97c410f400 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.889+0000 7f97baa00640 1 -- 172.21.15.103:0/224010900 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97bc004cc0 con 0x7f97c410f400 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.889+0000 7f97baa00640 1 -- 172.21.15.103:0/224010900 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc010720 con 0x7f97c410f400 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.889+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c41a1840 con 0x7f97c410f400 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.889+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c41a7f00 con 0x7f97c410f400 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.890+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c4101c30 con 0x7f97c410f400 2024-09-11T00:26:10.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.892+0000 7f97baa00640 1 -- 172.21.15.103:0/224010900 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f97bc005560 con 0x7f97c410f400 2024-09-11T00:26:10.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.894+0000 7f97baa00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97a80b53e0 0x7f97a80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:10.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.894+0000 7f97baa00640 1 -- 172.21.15.103:0/224010900 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f97bc0e6590 con 0x7f97c410f400 2024-09-11T00:26:10.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.894+0000 7f97c1e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97a80b53e0 0x7f97a80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:10.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.895+0000 7f97c1e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97a80b53e0 0x7f97a80b78a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97b4005ab0 tx=0x7f97b4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:10.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:10.896+0000 7f97baa00640 1 -- 172.21.15.103:0/224010900 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f97bc0ad060 con 0x7f97c410f400 2024-09-11T00:26:11.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.119+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97c419c5d0 con 0x7f97c410f400 2024-09-11T00:26:11.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.120+0000 7f97baa00640 1 -- 172.21.15.103:0/224010900 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11940 (secure 0 0 0) 0x7f97c419c5d0 con 0x7f97c410f400 2024-09-11T00:26:11.121 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:26:11.121 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":34,"btime":"2024-09-11T00:25:53:267970+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:25:52.343283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25550":{"gid":25550,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3202362307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3202362307},{"type":"v1","addr":"172.21.15.183:6835","nonce":3202362307}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:26:11.122 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 34 2024-09-11T00:26:11.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97a80b53e0 msgr2=0x7f97a80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97a80b53e0 0x7f97a80b78a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97b4005ab0 tx=0x7f97b4005a20 comp rx=0 tx=0).stop 2024-09-11T00:26:11.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c410f400 msgr2=0x7f97c41a0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c410f400 0x7f97c41a0d80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00eb40 tx=0x7f97bc00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 shutdown_connections 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97a80b53e0 0x7f97a80b78a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97c410f400 0x7f97c41a0d80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97c4102170 0x7f97c419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 --2- 172.21.15.103:0/224010900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97c4101850 0x7f97c419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 >> 172.21.15.103:0/224010900 conn(0x7f97c40fd700 msgr2=0x7f97c4111600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 shutdown_connections 2024-09-11T00:26:11.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:11.122+0000 7f97c8e00640 1 -- 172.21.15.103:0/224010900 wait complete. 2024-09-11T00:26:11.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:11 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/224010900' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:26:11.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:11 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/224010900' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:26:11.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:11 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/224010900' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:26:12.334 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --set-id 9f1734c5 --release --await' 2024-09-11T00:26:12.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --set-id 9f1734c5 --release --await 2024-09-11T00:26:12.588 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:12 smithi117 ceph-mon[35525]: pgmap v492: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2024-09-11T00:26:12.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:12 smithi103 ceph-mon[30578]: pgmap v492: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2024-09-11T00:26:12.682 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:12 smithi183 ceph-mon[35934]: pgmap v492: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 4.4 KiB/s wr, 15 op/s 2024-09-11T00:26:12.709 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 -- 172.21.15.103:0/3653091628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 msgr2=0x7ffadc111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 --2- 172.21.15.103:0/3653091628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 0x7ffadc111b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009ca0 tx=0x7ffad402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 -- 172.21.15.103:0/3653091628 shutdown_connections 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 --2- 172.21.15.103:0/3653091628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 0x7ffadc111b50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 --2- 172.21.15.103:0/3653091628 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffadc10a9d0 0x7ffadc10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 --2- 172.21.15.103:0/3653091628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc104fc0 0x7ffadc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 -- 172.21.15.103:0/3653091628 >> 172.21.15.103:0/3653091628 conn(0x7ffadc1009f0 msgr2=0x7ffadc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.353+0000 7ffae3200640 1 -- 172.21.15.103:0/3653091628 shutdown_connections 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.354+0000 7ffae3200640 1 -- 172.21.15.103:0/3653091628 wait complete. 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.354+0000 7ffae3200640 1 Processor -- start 2024-09-11T00:26:13.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffae3200640 1 -- start start 2024-09-11T00:26:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffae3200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffadc104fc0 0x7ffadc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffae3200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc10a9d0 0x7ffadc19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffae3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 0x7ffadc1a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffae3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc114f80 con 0x7ffadc10b350 2024-09-11T00:26:13.358 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffae3200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc114e00 con 0x7ffadc104fc0 2024-09-11T00:26:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffae3200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc115100 con 0x7ffadc10a9d0 2024-09-11T00:26:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffadb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc10a9d0 0x7ffadc19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffadb400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc10a9d0 0x7ffadc19b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:50984/0 (socket says 172.21.15.103:50984) 2024-09-11T00:26:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.355+0000 7ffadb400640 1 -- 172.21.15.103:0/4144521211 learned_addr learned my addr 172.21.15.103:0/4144521211 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffadbe00640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffadc104fc0 0x7ffadc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.359 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffae1200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 0x7ffadc1a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffadb400640 1 -- 172.21.15.103:0/4144521211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffadc104fc0 msgr2=0x7ffadc19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffadb400640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffadc104fc0 0x7ffadc19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffadb400640 1 -- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 msgr2=0x7ffadc1a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffadb400640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 0x7ffadc1a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffadb400640 1 -- 172.21.15.103:0/4144521211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffadc1a14e0 con 0x7ffadc10a9d0 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffae1200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 0x7ffadc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffadb400640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc10a9d0 0x7ffadc19b970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9f0 tx=0x7ffac800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.356+0000 7ffad8c00640 1 -- 172.21.15.103:0/4144521211 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac800cd80 con 0x7ffadc10a9d0 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.357+0000 7ffad8c00640 1 -- 172.21.15.103:0/4144521211 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffac80108e0 con 0x7ffadc10a9d0 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.357+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffadc1a17d0 con 0x7ffadc10a9d0 2024-09-11T00:26:13.360 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.357+0000 7ffad8c00640 1 -- 172.21.15.103:0/4144521211 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8018980 con 0x7ffadc10a9d0 2024-09-11T00:26:13.361 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.357+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffadc1a7e90 con 0x7ffadc10a9d0 2024-09-11T00:26:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 Processor -- start 2024-09-11T00:26:13.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 -- start start 2024-09-11T00:26:13.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b7ff0 0x55d43d9b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.363 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9c48a0 0x55d43d9c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d43d92f290 con 0x55d43d9c48a0 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d43d91e6c0 con 0x55d43d9b8910 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d43d9224d0 con 0x55d43d9b7ff0 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffad8c00640 1 -- 172.21.15.103:0/4144521211 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7ffac8010430 con 0x7ffadc10a9d0 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.360+0000 7ffae1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b7ff0 0x55d43d9b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:33716/0 (socket says 172.21.15.103:33716) 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffadbe00640 1 -- 172.21.15.103:0/3025616783 learned_addr learned my addr 172.21.15.103:0/3025616783 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffae1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b7ff0 0x55d43d9b83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:50992/0 (socket says 172.21.15.103:50992) 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffadb400640 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9c48a0 0x55d43d9c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffae1200640 1 -- 172.21.15.103:0/3025616783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 msgr2=0x55d43d9c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffae1200640 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffae1200640 1 -- 172.21.15.103:0/3025616783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9c48a0 msgr2=0x55d43d9c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:13.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffae1200640 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9c48a0 0x55d43d9c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffae1200640 1 -- 172.21.15.103:0/3025616783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d43d8ab460 con 0x55d43d9b7ff0 2024-09-11T00:26:13.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.361+0000 7ffae1200640 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b7ff0 0x55d43d9b83d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009ca0 tx=0x7ffad4009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60fdfdb602721f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.362+0000 7ffad1600640 1 -- 172.21.15.103:0/3025616783 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4031cf0 con 0x55d43d9b7ff0 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.362+0000 7ffad1600640 1 -- 172.21.15.103:0/3025616783 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad4040070 con 0x55d43d9b7ff0 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.362+0000 7ffad1600640 1 -- 172.21.15.103:0/3025616783 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad40042a0 con 0x55d43d9b7ff0 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.362+0000 7ffad8c00640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ffac00b52e0 0x7ffac00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3025616783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b7ff0 msgr2=0x55d43d9b83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffad8c00640 1 -- 172.21.15.103:0/4144521211 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ffac80e6360 con 0x7ffadc10a9d0 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b7ff0 0x55d43d9b83d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffad4009ca0 tx=0x7ffad4009360 comp rx=0 tx=0).stop 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffadbe00640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ffac00b52e0 0x7ffac00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3025616783 shutdown_connections 2024-09-11T00:26:13.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9c48a0 0x55d43d9c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3025616783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b7ff0 0x55d43d9b83d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3025616783 >> 172.21.15.103:0/3025616783 conn(0x55d43d9b5e20 msgr2=0x55d43d9b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3025616783 shutdown_connections 2024-09-11T00:26:13.367 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.363+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3025616783 wait complete. 2024-09-11T00:26:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.364+0000 7ffadbe00640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ffac00b52e0 0x7ffac00b77a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55d43d9bb170 tx=0x7ffacc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.364+0000 7ffae5bc0740 1 -- start start 2024-09-11T00:26:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.364+0000 7ffae5bc0740 1 Processor -- start 2024-09-11T00:26:13.368 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9c48a0 0x55d43d9c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d43d77fbb0 con 0x55d43d9b7ff0 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d43d8b96c0 con 0x55d43d9b8910 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d43d9c74f0 con 0x55d43d9c48a0 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae5bc0740 10 client.0 _ll_register_callbacks cb 0x55d43d8d4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9c48a0 0x55d43d9c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9c48a0 0x55d43d9c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51006/0 (socket says 172.21.15.103:51006) 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 -- 172.21.15.103:0/1339563801 learned_addr learned my addr 172.21.15.103:0/1339563801 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:55446/0 (socket says 172.21.15.103:55446) 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 -- 172.21.15.103:0/1339563801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 msgr2=0x55d43d9c1df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c1df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.369 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 -- 172.21.15.103:0/1339563801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 msgr2=0x55d43d9c18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.365+0000 7ffae1200640 1 -- 172.21.15.103:0/1339563801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d43d67cb00 con 0x55d43d9c48a0 2024-09-11T00:26:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.366+0000 7ffae1200640 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9c48a0 0x55d43d9c2330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad4031da0 tx=0x7ffad4031dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba9bb38f9e20d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.366+0000 7ffac6000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:26:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.366+0000 7ffac6000640 1 -- 172.21.15.103:0/1339563801 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad4051020 con 0x55d43d9c48a0 2024-09-11T00:26:13.370 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.366+0000 7ffac6000640 1 -- 172.21.15.103:0/1339563801 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ffad4031860 con 0x55d43d9c48a0 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.366+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad4040070 con 0x55d43d9c48a0 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffac6000640 1 -- 172.21.15.103:0/1339563801 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffad4009540 con 0x55d43d9c48a0 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 10 client.25741 fetch_fsmap learned FSMap version 34 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d43d6f3960 con 0x55d43d9c48a0 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffac6000640 1 -- 172.21.15.103:0/1339563801 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad4040070 con 0x55d43d9c48a0 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffac6000640 1 -- 172.21.15.103:0/1339563801 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7ffad4030860 con 0x55d43d9c48a0 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 10 client.25741 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 10 client.25741 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 20 client.25741 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:13.372 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9dca90 0x55d43d9dee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 4 client.25741 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.367+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d43d9311e0 con 0x55d43d9dca90 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.368+0000 7ffadbe00640 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9dca90 0x55d43d9dee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.368+0000 7ffadbe00640 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9dca90 0x55d43d9dee80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35058104249d10d7 server_cookie=88bd178bc9b788a0 in_seq=0 out_seq=0 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.368+0000 7ffac6000640 10 client.25741 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffac6000640 1 -- 172.21.15.103:0/1339563801 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d43d9311e0 con 0x55d43d9dca90 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffac6000640 10 client.25741 handle_command_reply: tid=0 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 client.25741 shutdown 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9c48a0 msgr2=0x55d43d9c2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9c48a0 0x55d43d9c2330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad4031da0 tx=0x7ffad4031dd0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 shutdown_connections 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9dca90 0x55d43d9dee80 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9c48a0 0x55d43d9c2330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.373 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9c1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/1339563801 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 >> 172.21.15.103:0/1339563801 conn(0x55d43d9b5e20 msgr2=0x55d43d9bb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.369+0000 7ffac6000640 0 client.25741 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.370+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 shutdown_connections 2024-09-11T00:26:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.370+0000 7ffae5bc0740 1 -- 172.21.15.103:0/1339563801 wait complete. 2024-09-11T00:26:13.374 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.370+0000 7ffae5bc0740 20 client.25741 trim_cache size 0 max 16384 2024-09-11T00:26:13.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 Processor -- start 2024-09-11T00:26:13.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 -- start start 2024-09-11T00:26:13.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9bb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1b0290 con 0x55d43d9b7ff0 2024-09-11T00:26:13.391 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1adaa0 con 0x55d43d9b8910 2024-09-11T00:26:13.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1ab910 con 0x55d43d9b5e20 2024-09-11T00:26:13.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.388+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9becd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51014/0 (socket says 172.21.15.103:51014) 2024-09-11T00:26:13.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 -- 172.21.15.103:0/3586599896 learned_addr learned my addr 172.21.15.103:0/3586599896 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:13.392 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffae1200640 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9bb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 -- 172.21.15.103:0/3586599896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 msgr2=0x55d43d9bb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9bb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 -- 172.21.15.103:0/3586599896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 msgr2=0x55d43d9c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.393 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 -- 172.21.15.103:0/3586599896 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d43d9b4c70 con 0x55d43d9b5e20 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadb400640 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffadbe00640 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9becd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffacc01d620 tx=0x7ffacc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c876c29bb7ed3190 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.389+0000 7ffad0c00640 1 -- 172.21.15.103:0/3586599896 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc01dd30 con 0x55d43d9b5e20 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffad0c00640 1 -- 172.21.15.103:0/3586599896 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffacc02b8e0 con 0x55d43d9b5e20 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffad0c00640 1 -- 172.21.15.103:0/3586599896 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc031a10 con 0x55d43d9b5e20 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3586599896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 msgr2=0x55d43d9becd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9becd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffacc01d620 tx=0x7ffacc01b000 comp rx=0 tx=0).stop 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3586599896 shutdown_connections 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b8910 0x55d43d9bb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b7ff0 0x55d43d9c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/3586599896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9becd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3586599896 >> 172.21.15.103:0/3586599896 conn(0x55d43d9bf3b0 msgr2=0x55d43d9c2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.390+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3586599896 shutdown_connections 2024-09-11T00:26:13.394 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.391+0000 7ffae5bc0740 1 -- 172.21.15.103:0/3586599896 wait complete. 2024-09-11T00:26:13.395 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.391+0000 7ffae5bc0740 1 -- start start 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.391+0000 7ffae5bc0740 1 Processor -- start 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b7ff0 0x55d43d9de810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffae5bc0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b8910 0x55d43d9dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1abf60 con 0x55d43d9b8910 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1ad3d0 con 0x55d43d9b7ff0 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffae5bc0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffadc1ac630 con 0x55d43d9b5e20 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffae5bc0740 10 client.0 _ll_register_callbacks cb 0x7ffacc0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffadbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9e22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51028/0 (socket says 172.21.15.103:51028) 2024-09-11T00:26:13.397 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffadbe00640 1 -- 172.21.15.103:0/2871874838 learned_addr learned my addr 172.21.15.103:0/2871874838 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffadb400640 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b7ff0 0x55d43d9de810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffadbe00640 1 -- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b7ff0 msgr2=0x55d43d9de810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffadbe00640 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b7ff0 0x55d43d9de810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.392+0000 7ffadbe00640 1 -- 172.21.15.103:0/2871874838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b8910 msgr2=0x55d43d9dedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffadbe00640 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b8910 0x55d43d9dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffadbe00640 1 -- 172.21.15.103:0/2871874838 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d43d9df5e0 con 0x55d43d9b5e20 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffadbe00640 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9e22e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffacc01dec0 tx=0x7ffacc012db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f55be9d6f562102 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffac6000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffac6000640 1 -- 172.21.15.103:0/2871874838 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffacc02b630 con 0x55d43d9b5e20 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffac6000640 1 -- 172.21.15.103:0/2871874838 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ffacc02ba90 con 0x55d43d9b5e20 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffacc02b8e0 con 0x55d43d9b5e20 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffac6000640 1 -- 172.21.15.103:0/2871874838 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffacc041050 con 0x55d43d9b5e20 2024-09-11T00:26:13.398 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffae5bc0740 10 client.25747 fetch_fsmap learned FSMap version 34 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.393+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d43da4a710 con 0x55d43d9b5e20 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffac6000640 1 -- 172.21.15.103:0/2871874838 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffacc02b8e0 con 0x55d43d9b5e20 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffac6000640 1 -- 172.21.15.103:0/2871874838 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7ffacc02f020 con 0x55d43d9b5e20 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffae5bc0740 10 client.25747 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffae5bc0740 10 client.25747 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffae5bc0740 20 client.25747 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9c48a0 0x55d43da4cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffae5bc0740 4 client.25747 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9f1734c5", "release": true, "await": true}] 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9f1734c5", "release": true, "await": true}) -- 0x55d43d92cc90 con 0x55d43d9c48a0 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.394+0000 7ffadb400640 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9c48a0 0x55d43da4cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.395+0000 7ffadb400640 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9c48a0 0x55d43da4cf10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333af3c779ca4604 server_cookie=12a814dfe744fd8a in_seq=0 out_seq=0 2024-09-11T00:26:13.399 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.395+0000 7ffac6000640 10 client.25747 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffac6000640 1 -- 172.21.15.103:0/2871874838 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d43d92cc90 con 0x55d43d9c48a0 2024-09-11T00:26:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffac6000640 10 client.25747 handle_command_reply: tid=0 2024-09-11T00:26:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 client.25747 shutdown 2024-09-11T00:26:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 msgr2=0x55d43d9e22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9e22e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffacc01dec0 tx=0x7ffacc012db0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 shutdown_connections 2024-09-11T00:26:13.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d43d9b8910 0x55d43d9dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d43d9b7ff0 0x55d43d9de810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d43d9c48a0 0x55d43da4cf10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 --2- 172.21.15.103:0/2871874838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d43d9b5e20 0x55d43d9e22e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.493+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 >> 172.21.15.103:0/2871874838 conn(0x55d43d9bf3b0 msgr2=0x55d43d9c3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.494+0000 7ffac6000640 0 client.25747 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.494+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 shutdown_connections 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.494+0000 7ffae5bc0740 1 -- 172.21.15.103:0/2871874838 wait complete. 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.494+0000 7ffae5bc0740 20 client.25747 trim_cache size 0 max 16384 2024-09-11T00:26:13.497 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 34, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 12, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "9f1734c5": { 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 12, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 140.5, 2024-09-11T00:26:13.498 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:26:13.499 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ffac00b52e0 msgr2=0x7ffac00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ffac00b52e0 0x7ffac00b77a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55d43d9bb170 tx=0x7ffacc005f90 comp rx=0 tx=0).stop 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc10a9d0 msgr2=0x7ffadc19b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc10a9d0 0x7ffadc19b970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e9f0 tx=0x7ffac800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 shutdown_connections 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7ffac00b52e0 0x7ffac00b77a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffadc10b350 0x7ffadc1a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffadc10a9d0 0x7ffadc19b970 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 --2- 172.21.15.103:0/4144521211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffadc104fc0 0x7ffadc19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:13.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 >> 172.21.15.103:0/4144521211 conn(0x7ffadc1009f0 msgr2=0x7ffadc102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:13.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 shutdown_connections 2024-09-11T00:26:13.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:26:13.495+0000 7ffae3200640 1 -- 172.21.15.103:0/4144521211 wait complete. 2024-09-11T00:26:14.113 INFO:tasks.workunit.client.0.smithi103.stdout:config.status: executing libtool commands 2024-09-11T00:26:14.297 INFO:tasks.workunit.client.0.smithi103.stdout:Building include 2024-09-11T00:26:14.297 INFO:tasks.workunit.client.0.smithi103.stdout:Building lib 2024-09-11T00:26:14.298 INFO:tasks.workunit.client.0.smithi103.stdout:Building tests 2024-09-11T00:26:14.307 INFO:tasks.workunit.client.0.smithi103.stdout: [LTDEP] 2024-09-11T00:26:14.311 INFO:tasks.workunit.client.0.smithi103.stdout:Building btrfs 2024-09-11T00:26:14.312 INFO:tasks.workunit.client.0.smithi103.stdout:Building ceph 2024-09-11T00:26:14.313 INFO:tasks.workunit.client.0.smithi103.stdout:Building cifs 2024-09-11T00:26:14.330 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-09-11T00:26:14.333 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-09-11T00:26:14.364 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-09-11T00:26:14.392 INFO:tasks.workunit.client.0.smithi103.stdout:Building ext4 2024-09-11T00:26:14.402 INFO:tasks.workunit.client.0.smithi103.stdout:Building f2fs 2024-09-11T00:26:14.417 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-09-11T00:26:14.424 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-09-11T00:26:14.482 INFO:tasks.workunit.client.0.smithi103.stdout:Building generic 2024-09-11T00:26:14.520 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9f1734c5', total seconds elapsed: 25.1 2024-09-11T00:26:14.609 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-09-11T00:26:14.628 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:14 smithi103 ceph-mon[30578]: pgmap v493: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-11T00:26:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:14 smithi117 ceph-mon[35525]: pgmap v493: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-11T00:26:14.696 INFO:tasks.workunit.client.0.smithi103.stdout:Building nfs 2024-09-11T00:26:14.710 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-09-11T00:26:14.756 INFO:tasks.workunit.client.0.smithi103.stdout:Building ocfs2 2024-09-11T00:26:14.756 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] dataascii.lo 2024-09-11T00:26:14.769 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-09-11T00:26:14.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:14 smithi183 ceph-mon[35934]: pgmap v493: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-11T00:26:14.814 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] databin.lo 2024-09-11T00:26:15.005 INFO:tasks.workunit.client.0.smithi103.stdout:Building overlay 2024-09-11T00:26:15.032 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-09-11T00:26:15.086 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] datapid.lo 2024-09-11T00:26:15.294 INFO:tasks.workunit.client.0.smithi103.stdout:Building perf 2024-09-11T00:26:15.307 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] file_lock.lo 2024-09-11T00:26:15.310 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-09-11T00:26:15.350 INFO:tasks.workunit.client.0.smithi103.stdout:Building selftest 2024-09-11T00:26:15.364 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-09-11T00:26:15.392 INFO:tasks.workunit.client.0.smithi103.stdout:Building shared 2024-09-11T00:26:15.406 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-09-11T00:26:15.415 INFO:tasks.workunit.client.0.smithi103.stdout:Building tmpfs 2024-09-11T00:26:15.431 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-09-11T00:26:15.470 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] forker.lo 2024-09-11T00:26:15.474 INFO:tasks.workunit.client.0.smithi103.stdout:Building udf 2024-09-11T00:26:15.489 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-09-11T00:26:15.527 INFO:tasks.workunit.client.0.smithi103.stdout:Building xfs 2024-09-11T00:26:15.584 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] pattern.lo 2024-09-11T00:26:15.678 INFO:tasks.workunit.client.0.smithi103.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-09-11T00:26:15.721 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] open_flags.lo 2024-09-11T00:26:15.812 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] random_range.lo 2024-09-11T00:26:15.984 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] string_to_tokens.lo 2024-09-11T00:26:16.176 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] str_to_bytes.lo 2024-09-11T00:26:16.191 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] tlibio.lo 2024-09-11T00:26:16.457 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] write_log.lo 2024-09-11T00:26:16.545 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] random.lo 2024-09-11T00:26:16.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:16 smithi103 ceph-mon[30578]: pgmap v494: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-11T00:26:16.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:16 smithi117 ceph-mon[35525]: pgmap v494: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-11T00:26:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:16 smithi183 ceph-mon[35934]: pgmap v494: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 6.5 KiB/s wr, 12 op/s 2024-09-11T00:26:16.824 INFO:tasks.workunit.client.0.smithi103.stdout: [LD] 2024-09-11T00:26:17.037 INFO:tasks.workunit.client.0.smithi103.stdout:Building ltp 2024-09-11T00:26:17.037 INFO:tasks.workunit.client.0.smithi103.stdout:Building src 2024-09-11T00:26:17.037 INFO:tasks.workunit.client.0.smithi103.stdout:Building m4 2024-09-11T00:26:17.049 INFO:tasks.workunit.client.0.smithi103.stdout: [DEP] 2024-09-11T00:26:17.051 INFO:tasks.workunit.client.0.smithi103.stdout:Building common 2024-09-11T00:26:17.068 INFO:tasks.workunit.client.0.smithi103.stdout: [DEP] 2024-09-11T00:26:17.476 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] doio 2024-09-11T00:26:17.476 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fsstress 2024-09-11T00:26:17.477 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fsx 2024-09-11T00:26:18.342 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] growfiles 2024-09-11T00:26:19.029 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:18 smithi183 ceph-mon[35934]: pgmap v495: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 KiB/s wr, 40 op/s 2024-09-11T00:26:19.029 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:18 smithi117 ceph-mon[35525]: pgmap v495: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 KiB/s wr, 40 op/s 2024-09-11T00:26:19.029 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] iogen 2024-09-11T00:26:19.030 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:18 smithi103 ceph-mon[30578]: pgmap v495: 129 pgs: 129 active+clean; 51 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 6.2 KiB/s wr, 40 op/s 2024-09-11T00:26:19.030 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-stress 2024-09-11T00:26:19.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:19 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:26:19.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:19 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:26:19.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:19 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:26:20.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:20 smithi103 ceph-mon[30578]: pgmap v496: 129 pgs: 129 active+clean; 52 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 36 KiB/s wr, 160 op/s 2024-09-11T00:26:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:20 smithi117 ceph-mon[35525]: pgmap v496: 129 pgs: 129 active+clean; 52 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 36 KiB/s wr, 160 op/s 2024-09-11T00:26:20.750 INFO:tasks.workunit.client.0.smithi103.stdout: [DEP] 2024-09-11T00:26:20.751 INFO:tasks.workunit.client.0.smithi103.stdout: [DEP] 2024-09-11T00:26:20.752 INFO:tasks.workunit.client.0.smithi103.stdout: [DEP] 2024-09-11T00:26:20.755 INFO:tasks.workunit.client.0.smithi103.stdout: [DEP] 2024-09-11T00:26:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:20 smithi183 ceph-mon[35934]: pgmap v496: 129 pgs: 129 active+clean; 52 MiB data, 616 MiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 36 KiB/s wr, 160 op/s 2024-09-11T00:26:21.185 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] dirstress 2024-09-11T00:26:21.185 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fill 2024-09-11T00:26:21.185 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fill2 2024-09-11T00:26:21.185 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] getpagesize 2024-09-11T00:26:21.468 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] holes 2024-09-11T00:26:21.500 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] lstat64 2024-09-11T00:26:21.591 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] permname 2024-09-11T00:26:21.591 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] nametest 2024-09-11T00:26:21.823 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] randholes 2024-09-11T00:26:21.825 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] runas 2024-09-11T00:26:21.930 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] truncfile 2024-09-11T00:26:22.034 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] usemem 2024-09-11T00:26:22.165 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] mmapcat 2024-09-11T00:26:22.223 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] append_reader 2024-09-11T00:26:22.269 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] append_writer 2024-09-11T00:26:22.356 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] dirperf 2024-09-11T00:26:22.449 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] metaperf 2024-09-11T00:26:22.532 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] devzero 2024-09-11T00:26:22.565 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] feature 2024-09-11T00:26:22.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:22 smithi103 ceph-mon[30578]: pgmap v497: 129 pgs: 129 active+clean; 52 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 109 KiB/s wr, 333 op/s 2024-09-11T00:26:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:22 smithi117 ceph-mon[35525]: pgmap v497: 129 pgs: 129 active+clean; 52 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 109 KiB/s wr, 333 op/s 2024-09-11T00:26:22.744 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] alloc 2024-09-11T00:26:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:22 smithi183 ceph-mon[35934]: pgmap v497: 129 pgs: 129 active+clean; 52 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 109 KiB/s wr, 333 op/s 2024-09-11T00:26:22.868 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fault 2024-09-11T00:26:22.883 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fstest 2024-09-11T00:26:22.933 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_access_root 2024-09-11T00:26:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.030+0000 7f1309c00640 1 -- 172.21.15.103:0/4269626658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130406b470 msgr2=0x7f1304105530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.030+0000 7f1309c00640 1 --2- 172.21.15.103:0/4269626658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130406b470 0x7f1304105530 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12f4009900 tx=0x7f12f402a400 comp rx=0 tx=0).stop 2024-09-11T00:26:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.031+0000 7f1309c00640 1 -- 172.21.15.103:0/4269626658 shutdown_connections 2024-09-11T00:26:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.031+0000 7f1309c00640 1 --2- 172.21.15.103:0/4269626658 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1304108670 0x7f130410ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.031+0000 7f1309c00640 1 --2- 172.21.15.103:0/4269626658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1304105ac0 0x7f1304107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.031+0000 7f1309c00640 1 --2- 172.21.15.103:0/4269626658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f130406b470 0x7f1304105530 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.031+0000 7f1309c00640 1 -- 172.21.15.103:0/4269626658 >> 172.21.15.103:0/4269626658 conn(0x7f13040fd140 msgr2=0x7f13040ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.032+0000 7f1309c00640 1 -- 172.21.15.103:0/4269626658 shutdown_connections 2024-09-11T00:26:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.032+0000 7f1309c00640 1 -- 172.21.15.103:0/4269626658 wait complete. 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.032+0000 7f1309c00640 1 Processor -- start 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1309c00640 1 -- start start 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1309c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130406b470 0x7f130419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1309c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304105ac0 0x7f130419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1309c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1304108670 0x7f130419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1309c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1304110800 con 0x7f1304108670 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1309c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1304110680 con 0x7f1304105ac0 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1309c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1304110980 con 0x7f130406b470 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1302a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130406b470 0x7f130419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1302000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304105ac0 0x7f130419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.033+0000 7f1302a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130406b470 0x7f130419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:53204/0 (socket says 172.21.15.103:53204) 2024-09-11T00:26:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304105ac0 0x7f130419b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52978/0 (socket says 172.21.15.103:52978) 2024-09-11T00:26:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302a00640 1 -- 172.21.15.103:0/1080890973 learned_addr learned my addr 172.21.15.103:0/1080890973 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1303400640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1304108670 0x7f130419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302000640 1 -- 172.21.15.103:0/1080890973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130406b470 msgr2=0x7f130419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302000640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130406b470 0x7f130419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302000640 1 -- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1304108670 msgr2=0x7f130419c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302000640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1304108670 0x7f130419c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302000640 1 -- 172.21.15.103:0/1080890973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130419cac0 con 0x7f1304105ac0 2024-09-11T00:26:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.034+0000 7f1302000640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304105ac0 0x7f130419b5d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005fd0 tx=0x7f12f000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.035+0000 7f12fb400640 1 -- 172.21.15.103:0/1080890973 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0009c30 con 0x7f1304105ac0 2024-09-11T00:26:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.035+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130419cdb0 con 0x7f1304105ac0 2024-09-11T00:26:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.035+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13041a3480 con 0x7f1304105ac0 2024-09-11T00:26:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.035+0000 7f12fb400640 1 -- 172.21.15.103:0/1080890973 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f0009dd0 con 0x7f1304105ac0 2024-09-11T00:26:23.038 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.035+0000 7f12fb400640 1 -- 172.21.15.103:0/1080890973 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0003e20 con 0x7f1304105ac0 2024-09-11T00:26:23.040 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.038+0000 7f130cbf6740 1 Processor -- start 2024-09-11T00:26:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.038+0000 7f130cbf6740 1 -- start start 2024-09-11T00:26:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee49b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4a0c0 0x5651fee4c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee4efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f130cbf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651fedba6b0 con 0x5651fee4cb00 2024-09-11T00:26:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f130cbf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651feda9ae0 con 0x5651fee47790 2024-09-11T00:26:23.041 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f130cbf6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651fedad8f0 con 0x5651fee4a0c0 2024-09-11T00:26:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f1303400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee49b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f1303400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee49b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52980/0 (socket says 172.21.15.103:52980) 2024-09-11T00:26:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f1303400640 1 -- 172.21.15.103:0/1774327429 learned_addr learned my addr 172.21.15.103:0/1774327429 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.039+0000 7f1302a00640 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4a0c0 0x5651fee4c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.040+0000 7f1302a00640 1 -- 172.21.15.103:0/1774327429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 msgr2=0x5651fee49b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:23.042 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.040+0000 7f1302a00640 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee49b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.040+0000 7f1302a00640 1 -- 172.21.15.103:0/1774327429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 msgr2=0x5651fee4efc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:26:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.040+0000 7f1302a00640 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee4efc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.040+0000 7f1302a00640 1 -- 172.21.15.103:0/1774327429 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fed76780 con 0x5651fee4a0c0 2024-09-11T00:26:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.040+0000 7f1303400640 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee49b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.040+0000 7f1302a00640 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4a0c0 0x5651fee4c4b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005940 tx=0x7f12f4031c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24f6b12f6bfaedcc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.041+0000 7f12efe00640 1 -- 172.21.15.103:0/1774327429 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4035030 con 0x5651fee4a0c0 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.041+0000 7f12efe00640 1 -- 172.21.15.103:0/1774327429 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f4040070 con 0x5651fee4a0c0 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.041+0000 7f12fb400640 1 -- 172.21.15.103:0/1080890973 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f12f0010460 con 0x7f1304105ac0 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.041+0000 7f12efe00640 1 -- 172.21.15.103:0/1774327429 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4003ab0 con 0x5651fee4a0c0 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.041+0000 7f130cbf6740 1 -- 172.21.15.103:0/1774327429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4a0c0 msgr2=0x5651fee4c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.041+0000 7f130cbf6740 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4a0c0 0x5651fee4c4b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f12f4005940 tx=0x7f12f4031c40 comp rx=0 tx=0).stop 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 -- 172.21.15.103:0/1774327429 shutdown_connections 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee4efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4a0c0 0x5651fee4c4b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 --2- 172.21.15.103:0/1774327429 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee49b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 -- 172.21.15.103:0/1774327429 >> 172.21.15.103:0/1774327429 conn(0x5651fee428e0 msgr2=0x5651fee43670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 -- 172.21.15.103:0/1774327429 shutdown_connections 2024-09-11T00:26:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 -- 172.21.15.103:0/1774327429 wait complete. 2024-09-11T00:26:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.042+0000 7f130cbf6740 1 -- start start 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 1 Processor -- start 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee50ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee51540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee52110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651fece1ab0 con 0x5651fee4cb00 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5651fed50570 con 0x5651fee4a0c0 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651fee4fa80 con 0x5651fee47790 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.043+0000 7f130cbf6740 10 client.0 _ll_register_callbacks cb 0x5651fed5f100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1303400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee50ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1303400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee50ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:53224/0 (socket says 172.21.15.103:53224) 2024-09-11T00:26:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1303400640 1 -- 172.21.15.103:0/767449243 learned_addr learned my addr 172.21.15.103:0/767449243 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1302000640 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee52110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1302a00640 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee51540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f12fb400640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12e80b5270 0x7f12e80b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1302a00640 1 -- 172.21.15.103:0/767449243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 msgr2=0x5651fee50ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1302a00640 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee50ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1302a00640 1 -- 172.21.15.103:0/767449243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 msgr2=0x5651fee52110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1302a00640 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee52110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.044+0000 7f1302a00640 1 -- 172.21.15.103:0/767449243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651feb0b790 con 0x5651fee4a0c0 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.045+0000 7f12fb400640 1 -- 172.21.15.103:0/1080890973 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f12f00e58d0 con 0x7f1304105ac0 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.045+0000 7f1302a00640 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee51540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12f4031500 tx=0x7f12f40317d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d53efbb0e19b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.045+0000 7f12ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:26:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.045+0000 7f1303400640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12e80b5270 0x7f12e80b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.045+0000 7f12ecc00640 1 -- 172.21.15.103:0/767449243 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4035050 con 0x5651fee4a0c0 2024-09-11T00:26:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.045+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f4040070 con 0x5651fee4a0c0 2024-09-11T00:26:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.045+0000 7f12ecc00640 1 -- 172.21.15.103:0/767449243 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f12f4004450 con 0x5651fee4a0c0 2024-09-11T00:26:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.046+0000 7f12ecc00640 1 -- 172.21.15.103:0/767449243 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f12f405c050 con 0x5651fee4a0c0 2024-09-11T00:26:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.046+0000 7f1303400640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12e80b5270 0x7f12e80b7730 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12fc01abb0 tx=0x7f12fc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.046+0000 7f130cbf6740 10 client.25721 fetch_fsmap learned FSMap version 35 2024-09-11T00:26:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.046+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5651feb84620 con 0x5651fee4a0c0 2024-09-11T00:26:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f12ecc00640 1 -- 172.21.15.103:0/767449243 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f4040070 con 0x5651fee4a0c0 2024-09-11T00:26:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f12ecc00640 1 -- 172.21.15.103:0/767449243 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f12f40457d0 con 0x5651fee4a0c0 2024-09-11T00:26:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f130cbf6740 10 client.25721 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:26:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f130cbf6740 10 client.25721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f130cbf6740 20 client.25721 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f130cbf6740 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651fee5c340 0x5651fee5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f130cbf6740 4 client.25721 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.047+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651fedbc600 con 0x5651fee5c340 2024-09-11T00:26:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.048+0000 7f1302000640 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651fee5c340 0x5651fee5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.048+0000 7f1302000640 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651fee5c340 0x5651fee5e730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa23ee0b91356d89 server_cookie=a914dd6f0d5a89b6 in_seq=0 out_seq=0 2024-09-11T00:26:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.048+0000 7f12ecc00640 10 client.25721 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.049+0000 7f12ecc00640 1 -- 172.21.15.103:0/767449243 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651fedbc600 con 0x5651fee5c340 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.049+0000 7f12ecc00640 10 client.25721 handle_command_reply: tid=0 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.049+0000 7f130cbf6740 1 client.25721 shutdown 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.049+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 msgr2=0x5651fee51540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee51540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f12f4031500 tx=0x7f12f40317d0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 shutdown_connections 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee52110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee51540 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651fee5c340 0x5651fee5e730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 --2- 172.21.15.103:0/767449243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee50ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 >> 172.21.15.103:0/767449243 conn(0x5651fee428e0 msgr2=0x5651fee41a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f12ecc00640 0 client.25721 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 shutdown_connections 2024-09-11T00:26:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 1 -- 172.21.15.103:0/767449243 wait complete. 2024-09-11T00:26:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.050+0000 7f130cbf6740 20 client.25721 trim_cache size 0 max 16384 2024-09-11T00:26:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.073+0000 7f130cbf6740 1 Processor -- start 2024-09-11T00:26:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.073+0000 7f130cbf6740 1 -- start start 2024-09-11T00:26:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee4cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4a0c0 0x5651fee4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4cb00 0x5651fee50ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f130cbf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12e4012550 con 0x5651fee4a0c0 2024-09-11T00:26:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f130cbf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12e400fd60 con 0x5651fee47790 2024-09-11T00:26:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f130cbf6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12e400dbb0 con 0x5651fee4cb00 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f1303400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4cb00 0x5651fee50ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f1302000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4a0c0 0x5651fee4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f1302a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee4cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f1303400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4cb00 0x5651fee50ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:53226/0 (socket says 172.21.15.103:53226) 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f1302a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee4cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53000/0 (socket says 172.21.15.103:53000) 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.074+0000 7f1303400640 1 -- 172.21.15.103:0/541735853 learned_addr learned my addr 172.21.15.103:0/541735853 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.075+0000 7f1303400640 1 -- 172.21.15.103:0/541735853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 msgr2=0x5651fee4cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.075+0000 7f1303400640 1 --2- 172.21.15.103:0/541735853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee4cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.075+0000 7f1303400640 1 -- 172.21.15.103:0/541735853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4a0c0 msgr2=0x5651fee4d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.075+0000 7f1303400640 1 --2- 172.21.15.103:0/541735853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4a0c0 0x5651fee4d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.075+0000 7f1303400640 1 -- 172.21.15.103:0/541735853 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651fee52480 con 0x5651fee4cb00 2024-09-11T00:26:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.075+0000 7f1303400640 1 --2- 172.21.15.103:0/541735853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4cb00 0x5651fee50ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f12fc010c60 tx=0x7f12fc022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa6bad057cb037a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.076+0000 7f12ef400640 1 -- 172.21.15.103:0/541735853 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc010e10 con 0x5651fee4cb00 2024-09-11T00:26:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.076+0000 7f12ef400640 1 -- 172.21.15.103:0/541735853 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12fc02a070 con 0x5651fee4cb00 2024-09-11T00:26:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.076+0000 7f12ef400640 1 -- 172.21.15.103:0/541735853 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc025a30 con 0x5651fee4cb00 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.077+0000 7f130cbf6740 1 -- 172.21.15.103:0/541735853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4cb00 msgr2=0x5651fee50ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.077+0000 7f130cbf6740 1 --2- 172.21.15.103:0/541735853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4cb00 0x5651fee50ea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f12fc010c60 tx=0x7f12fc022000 comp rx=0 tx=0).stop 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 -- 172.21.15.103:0/541735853 shutdown_connections 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 --2- 172.21.15.103:0/541735853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee4cb00 0x5651fee50ea0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 --2- 172.21.15.103:0/541735853 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4a0c0 0x5651fee4d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 --2- 172.21.15.103:0/541735853 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee47790 0x5651fee4cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 -- 172.21.15.103:0/541735853 >> 172.21.15.103:0/541735853 conn(0x5651fee428e0 msgr2=0x5651fee50060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 -- 172.21.15.103:0/541735853 shutdown_connections 2024-09-11T00:26:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 -- 172.21.15.103:0/541735853 wait complete. 2024-09-11T00:26:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.078+0000 7f130cbf6740 1 -- start start 2024-09-11T00:26:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 1 Processor -- start 2024-09-11T00:26:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee54a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12e400e220 con 0x5651fee4cb00 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f12e400f690 con 0x5651fee4a0c0 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12e400e8f0 con 0x5651fee47790 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f130cbf6740 10 client.0 _ll_register_callbacks cb 0x7f12fc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f1302000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f1302000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53010/0 (socket says 172.21.15.103:53010) 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f1302a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.079+0000 7f1302000640 1 -- 172.21.15.103:0/2040578184 learned_addr learned my addr 172.21.15.103:0/2040578184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f1302000640 1 -- 172.21.15.103:0/2040578184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 msgr2=0x5651fee587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f1302000640 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f1302000640 1 -- 172.21.15.103:0/2040578184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 msgr2=0x5651fee54a30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f1302000640 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee54a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f1302000640 1 -- 172.21.15.103:0/2040578184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651fee550a0 con 0x5651fee4a0c0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f1302a00640 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f1302000640 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee544f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000c00 tx=0x7f12f0012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b06927a84db3725a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f12ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.080+0000 7f12ecc00640 1 -- 172.21.15.103:0/2040578184 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0020070 con 0x5651fee4a0c0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12fc02a070 con 0x5651fee4a0c0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f12ecc00640 1 -- 172.21.15.103:0/2040578184 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f12f00a80c0 con 0x5651fee4a0c0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f12ecc00640 1 -- 172.21.15.103:0/2040578184 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f12f00abe50 con 0x5651fee4a0c0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 10 client.25727 fetch_fsmap learned FSMap version 35 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5651feeda330 con 0x5651fee4a0c0 2024-09-11T00:26:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f12ecc00640 1 -- 172.21.15.103:0/2040578184 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12fc02a070 con 0x5651fee4a0c0 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f12ecc00640 1 -- 172.21.15.103:0/2040578184 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f12f00a77a0 con 0x5651fee4a0c0 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 10 client.25727 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 10 client.25727 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 20 client.25727 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651feedacf0 0x5651feedd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 4 client.25727 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.081+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651fedb80b0 con 0x5651feedacf0 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.082+0000 7f1302a00640 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651feedacf0 0x5651feedd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.082+0000 7f12ecc00640 10 client.25727 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.082+0000 7f1302a00640 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651feedacf0 0x5651feedd110 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48388a13cf38da9 server_cookie=47040b93991a6ae in_seq=0 out_seq=0 2024-09-11T00:26:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.140+0000 7f12ecc00640 1 -- 172.21.15.103:0/2040578184 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5651fedb80b0 con 0x5651feedacf0 2024-09-11T00:26:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.140+0000 7f12ecc00640 10 client.25727 handle_command_reply: tid=0 2024-09-11T00:26:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.140+0000 7f130cbf6740 1 client.25727 shutdown 2024-09-11T00:26:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 msgr2=0x5651fee544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee544f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f12f0000c00 tx=0x7f12f0012000 comp rx=0 tx=0).stop 2024-09-11T00:26:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 shutdown_connections 2024-09-11T00:26:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651fee4cb00 0x5651fee54a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5651fee4a0c0 0x5651fee544f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5651feedacf0 0x5651feedd110 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 --2- 172.21.15.103:0/2040578184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651fee47790 0x5651fee587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 >> 172.21.15.103:0/2040578184 conn(0x5651fee428e0 msgr2=0x5651fee50e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.141+0000 7f12ecc00640 0 client.25727 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.142+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 shutdown_connections 2024-09-11T00:26:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.142+0000 7f130cbf6740 1 -- 172.21.15.103:0/2040578184 wait complete. 2024-09-11T00:26:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.142+0000 7f130cbf6740 20 client.25727 trim_cache size 0 max 16384 2024-09-11T00:26:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:26:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:26:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:26:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:26:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12e80b5270 msgr2=0x7f12e80b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12e80b5270 0x7f12e80b7730 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12fc01abb0 tx=0x7f12fc004ab0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304105ac0 msgr2=0x7f130419b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304105ac0 0x7f130419b5d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f12f0005fd0 tx=0x7f12f000dde0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 shutdown_connections 2024-09-11T00:26:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f12e80b5270 0x7f12e80b7730 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1304108670 0x7f130419c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1304105ac0 0x7f130419b5d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 --2- 172.21.15.103:0/1080890973 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f130406b470 0x7f130419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.144+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 >> 172.21.15.103:0/1080890973 conn(0x7f13040fd140 msgr2=0x7f1304103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.145+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 shutdown_connections 2024-09-11T00:26:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:23.145+0000 7f1309c00640 1 -- 172.21.15.103:0/1080890973 wait complete. 2024-09-11T00:26:23.171 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] godown 2024-09-11T00:26:23.239 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] resvtest 2024-09-11T00:26:23.269 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] writemod 2024-09-11T00:26:23.343 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] writev_on_pagefault 2024-09-11T00:26:23.486 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] makeextents 2024-09-11T00:26:23.540 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] itrash 2024-09-11T00:26:23.596 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] rename 2024-09-11T00:26:23.676 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] multi_open_unlink 2024-09-11T00:26:23.835 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] unwritten_sync 2024-09-11T00:26:23.849 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] genhashnames 2024-09-11T00:26:23.852 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_holes 2024-09-11T00:26:23.980 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_writev 2024-09-11T00:26:24.145 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_truncate_cmtime 2024-09-11T00:26:24.161 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] dirhash_collide 2024-09-11T00:26:24.214 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_rename_overwrite 2024-09-11T00:26:24.318 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] holetest 2024-09-11T00:26:24.448 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_truncate_self 2024-09-11T00:26:24.499 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] af_unix 2024-09-11T00:26:24.500 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_stale_pmd 2024-09-11T00:26:24.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:24 smithi103 ceph-mon[30578]: pgmap v498: 129 pgs: 129 active+clean; 53 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 133 KiB/s wr, 365 op/s 2024-09-11T00:26:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:24 smithi117 ceph-mon[35525]: pgmap v498: 129 pgs: 129 active+clean; 53 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 133 KiB/s wr, 365 op/s 2024-09-11T00:26:24.725 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_cow_race 2024-09-11T00:26:24.743 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_fallocate 2024-09-11T00:26:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:24 smithi183 ceph-mon[35934]: pgmap v498: 129 pgs: 129 active+clean; 53 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 133 KiB/s wr, 365 op/s 2024-09-11T00:26:24.833 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fsync-err 2024-09-11T00:26:24.834 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_write_ro 2024-09-11T00:26:25.021 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_ext4_dax_journal_corruption 2024-09-11T00:26:25.025 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_ext4_dax_inline_corruption 2024-09-11T00:26:25.133 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_ofd_locks 2024-09-11T00:26:25.168 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_collision 2024-09-11T00:26:25.322 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] mmap-write-concurrent 2024-09-11T00:26:25.340 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_get_file_time 2024-09-11T00:26:25.511 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_create_short_dirs 2024-09-11T00:26:25.530 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_create_long_dirs 2024-09-11T00:26:25.628 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_enospc 2024-09-11T00:26:25.636 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_writev_overlap 2024-09-11T00:26:25.822 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] checkpoint_journal 2024-09-11T00:26:25.830 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] mmap-rw-fault 2024-09-11T00:26:25.968 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] allocstale 2024-09-11T00:26:25.997 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_cow_memory_failure 2024-09-11T00:26:26.107 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fake-dump-rootino 2024-09-11T00:26:26.163 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] dio-buf-fault 2024-09-11T00:26:26.294 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] rewinddir-test 2024-09-11T00:26:26.301 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] readdir-while-renames 2024-09-11T00:26:26.410 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] xfsctl 2024-09-11T00:26:26.472 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] bstat 2024-09-11T00:26:26.616 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] getdevicesize 2024-09-11T00:26:26.616 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mtab 2024-09-11T00:26:26.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:26 smithi103 ceph-mon[30578]: pgmap v499: 129 pgs: 129 active+clean; 49 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 205 KiB/s wr, 389 op/s 2024-09-11T00:26:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:26 smithi117 ceph-mon[35525]: pgmap v499: 129 pgs: 129 active+clean; 49 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 205 KiB/s wr, 389 op/s 2024-09-11T00:26:26.760 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] preallo_rw_pattern_reader 2024-09-11T00:26:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:26 smithi183 ceph-mon[35934]: pgmap v499: 129 pgs: 129 active+clean; 49 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 205 KiB/s wr, 389 op/s 2024-09-11T00:26:26.820 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] preallo_rw_pattern_writer 2024-09-11T00:26:26.893 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] ftrunc 2024-09-11T00:26:26.960 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] trunc 2024-09-11T00:26:27.067 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fs_perms 2024-09-11T00:26:27.150 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] testx 2024-09-11T00:26:27.181 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] looptest 2024-09-11T00:26:27.289 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] locktest 2024-09-11T00:26:27.363 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] unwritten_mmap 2024-09-11T00:26:27.433 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] bulkstat_unlink_test 2024-09-11T00:26:27.531 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] deduperace 2024-09-11T00:26:27.670 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] bulkstat_unlink_test_modified 2024-09-11T00:26:27.772 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_dir_offset 2024-09-11T00:26:27.791 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_futimens 2024-09-11T00:26:27.901 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_immutable 2024-09-11T00:26:28.011 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] stale_handle 2024-09-11T00:26:28.066 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] pwrite_mmap_blocked 2024-09-11T00:26:28.084 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_dir_offset2 2024-09-11T00:26:28.334 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] seek_sanity_test 2024-09-11T00:26:28.334 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] seek_copy_test 2024-09-11T00:26:28.420 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_readdir_1 2024-09-11T00:26:28.642 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_readdir_2 2024-09-11T00:26:28.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:28 smithi103 ceph-mon[30578]: pgmap v500: 129 pgs: 129 active+clean; 49 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 202 KiB/s wr, 391 op/s 2024-09-11T00:26:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:28 smithi117 ceph-mon[35525]: pgmap v500: 129 pgs: 129 active+clean; 49 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 202 KiB/s wr, 391 op/s 2024-09-11T00:26:28.707 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fsync-tester 2024-09-11T00:26:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:28 smithi183 ceph-mon[35934]: pgmap v500: 129 pgs: 129 active+clean; 49 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 202 KiB/s wr, 391 op/s 2024-09-11T00:26:28.781 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] nsexec 2024-09-11T00:26:28.933 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] cloner 2024-09-11T00:26:28.983 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] renameat2 2024-09-11T00:26:29.121 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_getcwd 2024-09-11T00:26:29.123 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] e4compact 2024-09-11T00:26:29.311 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] test-nextquota 2024-09-11T00:26:29.334 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] punch-alternating 2024-09-11T00:26:29.466 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] attr-list-by-handle-cursor-test 2024-09-11T00:26:29.541 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] listxattr 2024-09-11T00:26:29.643 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] dio-interleaved 2024-09-11T00:26:29.677 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_dir_type 2024-09-11T00:26:29.781 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] dio-invalidate-cache 2024-09-11T00:26:29.822 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] stat_test 2024-09-11T00:26:29.955 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_encrypted_d_revalidate 2024-09-11T00:26:29.972 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] attr_replace_test 2024-09-11T00:26:30.142 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] swapon 2024-09-11T00:26:30.254 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] mkswap 2024-09-11T00:26:30.282 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_attr_corruption 2024-09-11T00:26:30.291 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_open_tmpfiles 2024-09-11T00:26:30.480 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fscrypt-crypt-util 2024-09-11T00:26:30.563 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] splice-test 2024-09-11T00:26:30.563 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] bulkstat_null_ocount 2024-09-11T00:26:30.625 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] chprojid_fail 2024-09-11T00:26:30.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:30 smithi103 ceph-mon[30578]: pgmap v501: 129 pgs: 129 active+clean; 49 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 205 KiB/s wr, 368 op/s 2024-09-11T00:26:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:30 smithi117 ceph-mon[35525]: pgmap v501: 129 pgs: 129 active+clean; 49 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 205 KiB/s wr, 368 op/s 2024-09-11T00:26:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:30 smithi183 ceph-mon[35934]: pgmap v501: 129 pgs: 129 active+clean; 49 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 205 KiB/s wr, 368 op/s 2024-09-11T00:26:30.849 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] detached_mounts_propagation 2024-09-11T00:26:30.866 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] ext4_resize 2024-09-11T00:26:30.957 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_readdir_3 2024-09-11T00:26:31.165 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] splice2pipe 2024-09-11T00:26:31.169 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] uuid_ioctl 2024-09-11T00:26:31.323 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_snapshot_deleted_subvolume 2024-09-11T00:26:31.429 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fiemap-fault 2024-09-11T00:26:31.456 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fiemap-tester 2024-09-11T00:26:31.491 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_stripealign 2024-09-11T00:26:31.627 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] open_by_handle 2024-09-11T00:26:31.725 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_mmap_dio 2024-09-11T00:26:31.798 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] xfsfind 2024-09-11T00:26:31.900 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] t_reflink_read_race 2024-09-11T00:26:32.020 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] fssum 2024-09-11T00:26:32.060 INFO:tasks.workunit.client.0.smithi103.stdout:Building log-writes 2024-09-11T00:26:32.075 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] replay-log 2024-09-11T00:26:32.181 INFO:tasks.workunit.client.0.smithi103.stdout:Building perf 2024-09-11T00:26:32.191 INFO:tasks.workunit.client.0.smithi103.stdout:Building aio-dio-regress 2024-09-11T00:26:32.206 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aiocp 2024-09-11T00:26:32.260 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-append-write-fallocate-race 2024-09-11T00:26:32.555 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-append-write-read-race 2024-09-11T00:26:32.564 INFO:tasks.workunit.client.0.smithi103.stdout:Building vfs 2024-09-11T00:26:32.573 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-cow-race 2024-09-11T00:26:32.577 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] vfstest 2024-09-11T00:26:32.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:32 smithi103 ceph-mon[30578]: pgmap v502: 129 pgs: 129 active+clean; 50 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 244 KiB/s wr, 260 op/s 2024-09-11T00:26:32.684 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] mount-idmapped 2024-09-11T00:26:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:32 smithi117 ceph-mon[35525]: pgmap v502: 129 pgs: 129 active+clean; 50 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 244 KiB/s wr, 260 op/s 2024-09-11T00:26:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:32 smithi183 ceph-mon[35934]: pgmap v502: 129 pgs: 129 active+clean; 50 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 244 KiB/s wr, 260 op/s 2024-09-11T00:26:32.844 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-cycle-write 2024-09-11T00:26:32.880 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-eof-race 2024-09-11T00:26:33.149 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-extend-stat 2024-09-11T00:26:33.209 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-fcntl-race 2024-09-11T00:26:33.265 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-hole-filling-race 2024-09-11T00:26:33.436 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-invalidate-failure 2024-09-11T00:26:33.509 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-invalidate-readahead 2024-09-11T00:26:33.542 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aiodio_sparse2 2024-09-11T00:26:33.718 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-subblock-eof-read 2024-09-11T00:26:33.808 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-dio-write-verify 2024-09-11T00:26:33.887 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-09-11T00:26:33.978 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-09-11T00:26:34.140 INFO:tasks.workunit.client.0.smithi103.stdout: [CC] aio-last-ref-held-by-io 2024-09-11T00:26:34.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:34 smithi103 ceph-mon[30578]: pgmap v503: 129 pgs: 129 active+clean; 51 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 247 KiB/s wr, 89 op/s 2024-09-11T00:26:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:34 smithi117 ceph-mon[35525]: pgmap v503: 129 pgs: 129 active+clean; 51 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 247 KiB/s wr, 89 op/s 2024-09-11T00:26:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:34 smithi183 ceph-mon[35934]: pgmap v503: 129 pgs: 129 active+clean; 51 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 247 KiB/s wr, 89 op/s 2024-09-11T00:26:35.064 INFO:tasks.workunit.client.0.smithi103.stderr:+ cd .. 2024-09-11T00:26:35.064 INFO:tasks.workunit.client.0.smithi103.stderr:+ cp xfstests-dev/ltp/fsx . 2024-09-11T00:26:35.070 INFO:tasks.workunit.client.0.smithi103.stderr:+ OPTIONS=-z 2024-09-11T00:26:35.070 INFO:tasks.workunit.client.0.smithi103.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-09-11T00:26:35.072 INFO:tasks.workunit.client.0.smithi103.stdout:Seed set to 1 2024-09-11T00:26:35.096 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-11T00:26:35.096 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-11T00:26:35.099 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-11T00:26:35.099 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-11T00:26:35.099 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support clone range, disabling! 2024-09-11T00:26:35.102 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-11T00:26:35.104 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero size read 2024-09-11T00:26:35.105 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support exchange range, disabling! 2024-09-11T00:26:35.105 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero size read 2024-09-11T00:26:35.105 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x81d54 2024-09-11T00:26:35.109 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xb86d7 2024-09-11T00:26:35.118 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xe627e 2024-09-11T00:26:35.167 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0xf3194 2024-09-11T00:26:35.198 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xf3ceb 2024-09-11T00:26:35.205 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0xf47e4 2024-09-11T00:26:35.326 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xfaccc 2024-09-11T00:26:35.906 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xfe423 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 -- 172.21.15.103:0/3432630018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c1038e0 msgr2=0x7fb87c103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 --2- 172.21.15.103:0/3432630018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c1038e0 0x7fb87c103d60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb868009890 tx=0x7fb86802a360 comp rx=0 tx=0).stop 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 -- 172.21.15.103:0/3432630018 shutdown_connections 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 --2- 172.21.15.103:0/3432630018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1042a0 0x7fb87c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 --2- 172.21.15.103:0/3432630018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c1038e0 0x7fb87c103d60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 --2- 172.21.15.103:0/3432630018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb87c102790 0x7fb87c102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 -- 172.21.15.103:0/3432630018 >> 172.21.15.103:0/3432630018 conn(0x7fb87c0fe030 msgr2=0x7fb87c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 -- 172.21.15.103:0/3432630018 shutdown_connections 2024-09-11T00:26:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.284+0000 7fb882000640 1 -- 172.21.15.103:0/3432630018 wait complete. 2024-09-11T00:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 Processor -- start 2024-09-11T00:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 -- start start 2024-09-11T00:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c102790 0x7fb87c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1038e0 0x7fb87c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb87c1042a0 0x7fb87c19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10e400 con 0x7fb87c1042a0 2024-09-11T00:26:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10e280 con 0x7fb87c1038e0 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb882000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c10e580 con 0x7fb87c102790 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb87be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1038e0 0x7fb87c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb881600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb87c1042a0 0x7fb87c19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb880c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c102790 0x7fb87c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb87be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1038e0 0x7fb87c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48458/0 (socket says 172.21.15.103:48458) 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.286+0000 7fb87be00640 1 -- 172.21.15.103:0/1465421656 learned_addr learned my addr 172.21.15.103:0/1465421656 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb87be00640 1 -- 172.21.15.103:0/1465421656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c102790 msgr2=0x7fb87c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb87be00640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c102790 0x7fb87c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb87be00640 1 -- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb87c1042a0 msgr2=0x7fb87c19c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb87be00640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb87c1042a0 0x7fb87c19c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb87be00640 1 -- 172.21.15.103:0/1465421656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c19c900 con 0x7fb87c1038e0 2024-09-11T00:26:36.290 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb87be00640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1038e0 0x7fb87c19b410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb868005940 tx=0x7fb868032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb879600640 1 -- 172.21.15.103:0/1465421656 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868040020 con 0x7fb87c1038e0 2024-09-11T00:26:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.287+0000 7fb879600640 1 -- 172.21.15.103:0/1465421656 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb868030de0 con 0x7fb87c1038e0 2024-09-11T00:26:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.288+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c19cb90 con 0x7fb87c1038e0 2024-09-11T00:26:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.288+0000 7fb879600640 1 -- 172.21.15.103:0/1465421656 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb868037040 con 0x7fb87c1038e0 2024-09-11T00:26:36.291 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.288+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1a32c0 con 0x7fb87c1038e0 2024-09-11T00:26:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.291+0000 7fb886664740 1 Processor -- start 2024-09-11T00:26:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.291+0000 7fb886664740 1 -- start start 2024-09-11T00:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb879600640 1 -- 172.21.15.103:0/1465421656 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb8680040d0 con 0x7fb87c1038e0 2024-09-11T00:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb886664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb886664740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9db690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb886664740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb886664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cc4a9496b0 con 0x55cc4a9d6970 2024-09-11T00:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb886664740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cc4a938ae0 con 0x55cc4a9d92a0 2024-09-11T00:26:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb886664740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc4a93c8f0 con 0x55cc4a9dbce0 2024-09-11T00:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb87be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb87be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9de1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51350/0 (socket says 172.21.15.103:51350) 2024-09-11T00:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb87be00640 1 -- 172.21.15.103:0/2252548342 learned_addr learned my addr 172.21.15.103:0/2252548342 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb881600640 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.295 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.292+0000 7fb880c00640 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9db690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb87be00640 1 -- 172.21.15.103:0/2252548342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 msgr2=0x55cc4a9db690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb87be00640 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9db690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb87be00640 1 -- 172.21.15.103:0/2252548342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 msgr2=0x55cc4a9d8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb87be00640 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9d8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb87be00640 1 -- 172.21.15.103:0/2252548342 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4a905780 con 0x55cc4a9dbce0 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb880c00640 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9db690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb87be00640 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9de1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb868109e50 tx=0x7fb868109f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=988ff546c422a38f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb86dc00640 1 -- 172.21.15.103:0/2252548342 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86810ecb0 con 0x55cc4a9dbce0 2024-09-11T00:26:36.300 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb86dc00640 1 -- 172.21.15.103:0/2252548342 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb868113070 con 0x55cc4a9dbce0 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.293+0000 7fb86dc00640 1 -- 172.21.15.103:0/2252548342 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8681065e0 con 0x55cc4a9dbce0 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 -- 172.21.15.103:0/2252548342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 msgr2=0x55cc4a9de1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9de1a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb868109e50 tx=0x7fb868109f40 comp rx=0 tx=0).stop 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 -- 172.21.15.103:0/2252548342 shutdown_connections 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9de1a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9db690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 --2- 172.21.15.103:0/2252548342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9d8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 -- 172.21.15.103:0/2252548342 >> 172.21.15.103:0/2252548342 conn(0x55cc4a9d1ac0 msgr2=0x55cc4a9d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.294+0000 7fb886664740 1 -- 172.21.15.103:0/2252548342 shutdown_connections 2024-09-11T00:26:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.295+0000 7fb886664740 1 -- 172.21.15.103:0/2252548342 wait complete. 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.295+0000 7fb879600640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8640b51f0 0x7fb8640b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.295+0000 7fb879600640 1 -- 172.21.15.103:0/1465421656 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb868105ed0 con 0x7fb87c1038e0 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.295+0000 7fb880c00640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8640b51f0 0x7fb8640b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.295+0000 7fb886664740 1 -- start start 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.295+0000 7fb886664740 1 Processor -- start 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb880c00640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8640b51f0 0x7fb8640b76b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb8700096c0 tx=0x7fb870009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb886664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb886664740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb886664740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb886664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cc4a870ab0 con 0x55cc4a9d6970 2024-09-11T00:26:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb886664740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cc4a8df570 con 0x55cc4a9dbce0 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb886664740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cc4a9dea80 con 0x55cc4a9d92a0 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb886664740 10 client.0 _ll_register_callbacks cb 0x55cc4a8ee100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb87be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb881600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb880c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb87be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51360/0 (socket says 172.21.15.103:51360) 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb87be00640 1 -- 172.21.15.103:0/3065663143 learned_addr learned my addr 172.21.15.103:0/3065663143 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb881600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48484/0 (socket says 172.21.15.103:48484) 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb881600640 1 -- 172.21.15.103:0/3065663143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d92a0 msgr2=0x55cc4a9e0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb881600640 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb881600640 1 -- 172.21.15.103:0/3065663143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 msgr2=0x55cc4a9dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb881600640 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.296+0000 7fb881600640 1 -- 172.21.15.103:0/3065663143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4a69a790 con 0x55cc4a9dbce0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb881600640 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e0ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb874005fd0 tx=0x7fb874011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24c81b93082f526 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb862000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb862000640 1 -- 172.21.15.103:0/3065663143 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87400fcb0 con 0x55cc4a9dbce0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb862000640 1 -- 172.21.15.103:0/3065663143 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb874004b20 con 0x55cc4a9dbce0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb868113070 con 0x55cc4a9dbce0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb862000640 1 -- 172.21.15.103:0/3065663143 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb874025050 con 0x55cc4a9dbce0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb886664740 10 client.25739 fetch_fsmap learned FSMap version 35 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.297+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cc4a713620 con 0x55cc4a9dbce0 2024-09-11T00:26:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.298+0000 7fb862000640 1 -- 172.21.15.103:0/3065663143 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb868113070 con 0x55cc4a9dbce0 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb862000640 1 -- 172.21.15.103:0/3065663143 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fb874019070 con 0x55cc4a9dbce0 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb886664740 10 client.25739 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb886664740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb886664740 20 client.25739 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb886664740 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4a9eb720 0x55cc4a9edb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb886664740 4 client.25739 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc4a94b600 con 0x55cc4a9eb720 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.299+0000 7fb880c00640 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4a9eb720 0x55cc4a9edb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.300+0000 7fb880c00640 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4a9eb720 0x55cc4a9edb50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8805fd98e282f04c server_cookie=46988e40f8e5b0cd in_seq=0 out_seq=0 2024-09-11T00:26:36.307 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.300+0000 7fb862000640 10 client.25739 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb862000640 1 -- 172.21.15.103:0/3065663143 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc4a94b600 con 0x55cc4a9eb720 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb862000640 10 client.25739 handle_command_reply: tid=0 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 client.25739 shutdown 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 msgr2=0x55cc4a9e0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e0ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb874005fd0 tx=0x7fb874011e40 comp rx=0 tx=0).stop 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 shutdown_connections 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e0ff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4a9eb720 0x55cc4a9edb50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.308 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 --2- 172.21.15.103:0/3065663143 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 >> 172.21.15.103:0/3065663143 conn(0x55cc4a9d1ac0 msgr2=0x55cc4a9d0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb862000640 0 client.25739 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 shutdown_connections 2024-09-11T00:26:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 1 -- 172.21.15.103:0/3065663143 wait complete. 2024-09-11T00:26:36.309 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.301+0000 7fb886664740 20 client.25739 trim_cache size 0 max 16384 2024-09-11T00:26:36.322 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.320+0000 7fb886664740 1 Processor -- start 2024-09-11T00:26:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.320+0000 7fb886664740 1 -- start start 2024-09-11T00:26:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb886664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb886664740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9dc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb886664740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9dc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb886664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1affe0 con 0x55cc4a9d6970 2024-09-11T00:26:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb886664740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ad7f0 con 0x55cc4a9d92a0 2024-09-11T00:26:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb886664740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ab640 con 0x55cc4a9dbce0 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9dc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb881600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9dc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9dc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48498/0 (socket says 172.21.15.103:48498) 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 -- 172.21.15.103:0/3590546003 learned_addr learned my addr 172.21.15.103:0/3590546003 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb880c00640 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 -- 172.21.15.103:0/3590546003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 msgr2=0x55cc4a9dc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9dc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 -- 172.21.15.103:0/3590546003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 msgr2=0x55cc4a9ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb87be00640 1 -- 172.21.15.103:0/3590546003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc4a9d2d20 con 0x55cc4a9d92a0 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.321+0000 7fb881600640 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9dc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.322+0000 7fb87be00640 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9dc130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb8681039b0 tx=0x7fb868005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200ff26dedaac056 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.322+0000 7fb86d200640 1 -- 172.21.15.103:0/3590546003 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680c8950 con 0x55cc4a9d92a0 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.322+0000 7fb86d200640 1 -- 172.21.15.103:0/3590546003 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb868113070 con 0x55cc4a9d92a0 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.322+0000 7fb86d200640 1 -- 172.21.15.103:0/3590546003 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680cba70 con 0x55cc4a9d92a0 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 -- 172.21.15.103:0/3590546003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 msgr2=0x55cc4a9dc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9dc130 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb8681039b0 tx=0x7fb868005f90 comp rx=0 tx=0).stop 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 -- 172.21.15.103:0/3590546003 shutdown_connections 2024-09-11T00:26:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9dc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9dc130 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 --2- 172.21.15.103:0/3590546003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d6970 0x55cc4a9ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 -- 172.21.15.103:0/3590546003 >> 172.21.15.103:0/3590546003 conn(0x55cc4a9d1ac0 msgr2=0x55cc4a9d5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 -- 172.21.15.103:0/3590546003 shutdown_connections 2024-09-11T00:26:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 -- 172.21.15.103:0/3590546003 wait complete. 2024-09-11T00:26:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.323+0000 7fb886664740 1 -- start start 2024-09-11T00:26:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 1 Processor -- start 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d6970 0x55cc4a9e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1abcb0 con 0x55cc4a9d92a0 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ad120 con 0x55cc4a9dbce0 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ac380 con 0x55cc4a9d6970 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb886664740 10 client.0 _ll_register_callbacks cb 0x7fb874009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb880c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d6970 0x55cc4a9e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb880c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d6970 0x55cc4a9e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51378/0 (socket says 172.21.15.103:51378) 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb880c00640 1 -- 172.21.15.103:0/255172412 learned_addr learned my addr 172.21.15.103:0/255172412 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.324+0000 7fb87be00640 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb881600640 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb880c00640 1 -- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 msgr2=0x55cc4a9e5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb880c00640 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb880c00640 1 -- 172.21.15.103:0/255172412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d92a0 msgr2=0x55cc4a9e7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb880c00640 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb880c00640 1 -- 172.21.15.103:0/255172412 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc4a9d90d0 con 0x55cc4a9d6970 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb880c00640 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d6970 0x55cc4a9e6b30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb870005fd0 tx=0x7fb8700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c18ff84c9faf7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb862000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb862000640 1 -- 172.21.15.103:0/255172412 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87001cd10 con 0x55cc4a9d6970 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb862000640 1 -- 172.21.15.103:0/255172412 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb87002a400 con 0x55cc4a9d6970 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.325+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb868113070 con 0x55cc4a9d6970 2024-09-11T00:26:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.326+0000 7fb862000640 1 -- 172.21.15.103:0/255172412 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb87003d050 con 0x55cc4a9d6970 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.326+0000 7fb886664740 10 client.25783 fetch_fsmap learned FSMap version 35 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.326+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cc4aa61390 con 0x55cc4a9d6970 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.326+0000 7fb862000640 1 -- 172.21.15.103:0/255172412 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb868113070 con 0x55cc4a9d6970 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb862000640 1 -- 172.21.15.103:0/255172412 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fb87002f6c0 con 0x55cc4a9d6970 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb886664740 10 client.25783 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb886664740 10 client.25783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb886664740 20 client.25783 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb886664740 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4aa61d10 0x55cc4aa64130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb886664740 4 client.25783 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc4a9470b0 con 0x55cc4aa61d10 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb87be00640 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4aa61d10 0x55cc4aa64130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb87be00640 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4aa61d10 0x55cc4aa64130 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c83b3e7c3222860 server_cookie=de2252e6c65ba01e in_seq=0 out_seq=0 2024-09-11T00:26:36.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.327+0000 7fb862000640 10 client.25783 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.354+0000 7fb862000640 1 -- 172.21.15.103:0/255172412 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc4a9470b0 con 0x55cc4aa61d10 2024-09-11T00:26:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.354+0000 7fb862000640 10 client.25783 handle_command_reply: tid=0 2024-09-11T00:26:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.354+0000 7fb886664740 1 client.25783 shutdown 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.354+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d6970 msgr2=0x55cc4a9e6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.354+0000 7fb886664740 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d6970 0x55cc4a9e6b30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb870005fd0 tx=0x7fb8700046c0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 shutdown_connections 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cc4a9dbce0 0x55cc4a9e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc4a9d92a0 0x55cc4a9e7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cc4aa61d10 0x55cc4aa64130 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 --2- 172.21.15.103:0/255172412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cc4a9d6970 0x55cc4a9e6b30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb862000640 0 client.25783 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 >> 172.21.15.103:0/255172412 conn(0x55cc4a9d1ac0 msgr2=0x55cc4a9d5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 shutdown_connections 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 1 -- 172.21.15.103:0/255172412 wait complete. 2024-09-11T00:26:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.355+0000 7fb886664740 20 client.25783 trim_cache size 0 max 16384 2024-09-11T00:26:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:26:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:26:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:26:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8640b51f0 msgr2=0x7fb8640b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8640b51f0 0x7fb8640b76b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb8700096c0 tx=0x7fb870009260 comp rx=0 tx=0).stop 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1038e0 msgr2=0x7fb87c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1038e0 0x7fb87c19b410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb868005940 tx=0x7fb868032d50 comp rx=0 tx=0).stop 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 shutdown_connections 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb8640b51f0 0x7fb8640b76b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb87c1042a0 0x7fb87c19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c1038e0 0x7fb87c19b410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 --2- 172.21.15.103:0/1465421656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb87c102790 0x7fb87c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.356+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 >> 172.21.15.103:0/1465421656 conn(0x7fb87c0fe030 msgr2=0x7fb87c075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.357+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 shutdown_connections 2024-09-11T00:26:36.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:36.357+0000 7fb882000640 1 -- 172.21.15.103:0/1465421656 wait complete. 2024-09-11T00:26:36.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:36 smithi103 ceph-mon[30578]: pgmap v504: 129 pgs: 129 active+clean; 53 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 418 KiB/s wr, 71 op/s 2024-09-11T00:26:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:36 smithi117 ceph-mon[35525]: pgmap v504: 129 pgs: 129 active+clean; 53 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 418 KiB/s wr, 71 op/s 2024-09-11T00:26:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:36 smithi183 ceph-mon[35934]: pgmap v504: 129 pgs: 129 active+clean; 53 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 418 KiB/s wr, 71 op/s 2024-09-11T00:26:38.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:38 smithi103 ceph-mon[30578]: pgmap v505: 129 pgs: 129 active+clean; 53 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-09-11T00:26:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:38 smithi117 ceph-mon[35525]: pgmap v505: 129 pgs: 129 active+clean; 53 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-09-11T00:26:38.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:38 smithi183 ceph-mon[35934]: pgmap v505: 129 pgs: 129 active+clean; 53 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 346 KiB/s wr, 47 op/s 2024-09-11T00:26:39.270 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xfe492 2024-09-11T00:26:40.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:40 smithi103 ceph-mon[30578]: pgmap v506: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 346 KiB/s wr, 46 op/s 2024-09-11T00:26:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:40 smithi117 ceph-mon[35525]: pgmap v506: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 346 KiB/s wr, 46 op/s 2024-09-11T00:26:40.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:40 smithi183 ceph-mon[35934]: pgmap v506: 129 pgs: 129 active+clean; 53 MiB data, 669 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 346 KiB/s wr, 46 op/s 2024-09-11T00:26:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:41 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:26:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:41 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:26:41.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:41 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:26:42.263 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xff038 2024-09-11T00:26:42.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:42 smithi103 ceph-mon[30578]: pgmap v507: 129 pgs: 129 active+clean; 55 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-09-11T00:26:42.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:26:42.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:26:42.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:26:42.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:42 smithi183 ceph-mon[35934]: pgmap v507: 129 pgs: 129 active+clean; 55 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-09-11T00:26:42.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:26:42.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:26:42.681 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:26:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:42 smithi117 ceph-mon[35525]: pgmap v507: 129 pgs: 129 active+clean; 55 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.5 MiB/s wr, 222 op/s 2024-09-11T00:26:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:26:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:26:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:26:43.584 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0xffcac 2024-09-11T00:26:43.774 INFO:tasks.workunit.client.0.smithi103.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-09-11T00:26:44.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:44 smithi103 ceph-mon[30578]: pgmap v508: 129 pgs: 129 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-09-11T00:26:44.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:44 smithi117 ceph-mon[35525]: pgmap v508: 129 pgs: 129 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-09-11T00:26:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:44 smithi183 ceph-mon[35934]: pgmap v508: 129 pgs: 129 active+clean; 55 MiB data, 710 MiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 1.4 MiB/s wr, 212 op/s 2024-09-11T00:26:46.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:46 smithi103 ceph-mon[30578]: pgmap v509: 129 pgs: 129 active+clean; 57 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 423 op/s 2024-09-11T00:26:46.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:46 smithi117 ceph-mon[35525]: pgmap v509: 129 pgs: 129 active+clean; 57 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 423 op/s 2024-09-11T00:26:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:46 smithi183 ceph-mon[35934]: pgmap v509: 129 pgs: 129 active+clean; 57 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 423 op/s 2024-09-11T00:26:48.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:48 smithi103 ceph-mon[30578]: pgmap v510: 129 pgs: 129 active+clean; 57 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 408 op/s 2024-09-11T00:26:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:48 smithi117 ceph-mon[35525]: pgmap v510: 129 pgs: 129 active+clean; 57 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 408 op/s 2024-09-11T00:26:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:48 smithi183 ceph-mon[35934]: pgmap v510: 129 pgs: 129 active+clean; 57 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 408 op/s 2024-09-11T00:26:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 -- 172.21.15.103:0/3790704238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f946006b470 msgr2=0x7f9460105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 --2- 172.21.15.103:0/3790704238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f946006b470 0x7f9460105360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9454009900 tx=0x7f945402a410 comp rx=0 tx=0).stop 2024-09-11T00:26:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 -- 172.21.15.103:0/3790704238 shutdown_connections 2024-09-11T00:26:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 --2- 172.21.15.103:0/3790704238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94601084a0 0x7f946010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 --2- 172.21.15.103:0/3790704238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94601058f0 0x7f9460107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 --2- 172.21.15.103:0/3790704238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f946006b470 0x7f9460105360 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 -- 172.21.15.103:0/3790704238 >> 172.21.15.103:0/3790704238 conn(0x7f94600fcf30 msgr2=0x7f94600ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 -- 172.21.15.103:0/3790704238 shutdown_connections 2024-09-11T00:26:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.493+0000 7f9466a00640 1 -- 172.21.15.103:0/3790704238 wait complete. 2024-09-11T00:26:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.494+0000 7f9466a00640 1 Processor -- start 2024-09-11T00:26:49.497 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.494+0000 7f9466a00640 1 -- start start 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.494+0000 7f9466a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f946006b470 0x7f946019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9466a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94601058f0 0x7f946019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9466a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94601084a0 0x7f946019c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9466a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f946010e3d0 con 0x7f94601084a0 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9466a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f946010e250 con 0x7f946006b470 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9466a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f946010e550 con 0x7f94601058f0 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f946006b470 0x7f946019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f946006b470 0x7f946019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53824/0 (socket says 172.21.15.103:53824) 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 -- 172.21.15.103:0/1168262664 learned_addr learned my addr 172.21.15.103:0/1168262664 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9466000640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94601084a0 0x7f946019c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9464c00640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94601058f0 0x7f946019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.498 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 -- 172.21.15.103:0/1168262664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94601058f0 msgr2=0x7f946019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94601058f0 0x7f946019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 -- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94601084a0 msgr2=0x7f946019c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94601084a0 0x7f946019c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9465600640 1 -- 172.21.15.103:0/1168262664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946019c910 con 0x7f946006b470 2024-09-11T00:26:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.495+0000 7f9466000640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94601084a0 0x7f946019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:49.499 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.496+0000 7f9465600640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f946006b470 0x7f946019add0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9454005940 tx=0x7f94540042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.496+0000 7f945e000640 1 -- 172.21.15.103:0/1168262664 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945403f030 con 0x7f946006b470 2024-09-11T00:26:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.496+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946019cba0 con 0x7f946006b470 2024-09-11T00:26:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.496+0000 7f945e000640 1 -- 172.21.15.103:0/1168262664 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9454003cb0 con 0x7f946006b470 2024-09-11T00:26:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.496+0000 7f945e000640 1 -- 172.21.15.103:0/1168262664 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454035070 con 0x7f946006b470 2024-09-11T00:26:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.496+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a32d0 con 0x7f946006b470 2024-09-11T00:26:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.499+0000 7f946aea5740 1 Processor -- start 2024-09-11T00:26:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f946aea5740 1 -- start start 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f946aea5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53ea4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f946aea5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53ea7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f946aea5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eaa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f946aea5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf53e156b0 con 0x55bf53ea52a0 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f946aea5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf53e04ae0 con 0x55bf53ea2970 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f946aea5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bf53e088f0 con 0x55bf53ea7ce0 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f9466000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53ea4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f9466000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53ea4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53834/0 (socket says 172.21.15.103:53834) 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f9466000640 1 -- 172.21.15.103:0/3842396087 learned_addr learned my addr 172.21.15.103:0/3842396087 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f9464c00640 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eaa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.500+0000 7f945e000640 1 -- 172.21.15.103:0/1168262664 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f945403bbe0 con 0x7f946006b470 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9466000640 1 -- 172.21.15.103:0/3842396087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 msgr2=0x55bf53eaa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9466000640 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eaa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9466000640 1 -- 172.21.15.103:0/3842396087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 msgr2=0x55bf53ea7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9465600640 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53ea7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9466000640 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53ea7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9466000640 1 -- 172.21.15.103:0/3842396087 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf53dd1780 con 0x55bf53ea2970 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9465600640 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53ea7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9466000640 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53ea4d60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9458011760 tx=0x7f9458011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed7967e9c559389 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9452600640 1 -- 172.21.15.103:0/3842396087 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458004d20 con 0x55bf53ea2970 2024-09-11T00:26:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9452600640 1 -- 172.21.15.103:0/3842396087 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945800f9a0 con 0x55bf53ea2970 2024-09-11T00:26:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.501+0000 7f9452600640 1 -- 172.21.15.103:0/3842396087 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458004430 con 0x55bf53ea2970 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f946aea5740 1 -- 172.21.15.103:0/3842396087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 msgr2=0x55bf53ea4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f946aea5740 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53ea4d60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9458011760 tx=0x7f9458011c20 comp rx=0 tx=0).stop 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f945e000640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f94480b53e0 0x7f94480b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f9466000640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f94480b53e0 0x7f94480b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f945e000640 1 -- 172.21.15.103:0/1168262664 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f94541065e0 con 0x7f946006b470 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f946aea5740 1 -- 172.21.15.103:0/3842396087 shutdown_connections 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f946aea5740 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eaa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f946aea5740 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53ea7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.508 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f946aea5740 1 --2- 172.21.15.103:0/3842396087 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53ea4d60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.502+0000 7f946aea5740 1 -- 172.21.15.103:0/3842396087 >> 172.21.15.103:0/3842396087 conn(0x55bf53e9dac0 msgr2=0x55bf53e9e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.503+0000 7f946aea5740 1 -- 172.21.15.103:0/3842396087 shutdown_connections 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.503+0000 7f946aea5740 1 -- 172.21.15.103:0/3842396087 wait complete. 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.503+0000 7f9466000640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f94480b53e0 0x7f94480b78a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9458011760 tx=0x7f9458000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.503+0000 7f946aea5740 1 -- start start 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 1 Processor -- start 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53eabfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53eac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53ead250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bf53d3cab0 con 0x55bf53ea52a0 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55bf53dab570 con 0x55bf53ea2970 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bf53eaacc0 con 0x55bf53ea7ce0 2024-09-11T00:26:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f946aea5740 10 client.0 _ll_register_callbacks cb 0x55bf53dba100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f9465600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53eabfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f9466000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53ead250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f9465600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53eabfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53842/0 (socket says 172.21.15.103:53842) 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.504+0000 7f9465600640 1 -- 172.21.15.103:0/693230187 learned_addr learned my addr 172.21.15.103:0/693230187 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9464c00640 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53eac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9465600640 1 -- 172.21.15.103:0/693230187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 msgr2=0x55bf53ead250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9465600640 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53ead250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9465600640 1 -- 172.21.15.103:0/693230187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 msgr2=0x55bf53eac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9465600640 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53eac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9465600640 1 -- 172.21.15.103:0/693230187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf53b66790 con 0x55bf53ea2970 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9465600640 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53eabfe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94540cc760 tx=0x7f94540cc3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403712393e3fb7f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9446a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:26:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9446a00640 1 -- 172.21.15.103:0/693230187 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454049070 con 0x55bf53ea2970 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f945800f9a0 con 0x55bf53ea2970 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9446a00640 1 -- 172.21.15.103:0/693230187 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f94540c8cd0 con 0x55bf53ea2970 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f9446a00640 1 -- 172.21.15.103:0/693230187 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9454003e60 con 0x55bf53ea2970 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f946aea5740 10 client.25757 fetch_fsmap learned FSMap version 35 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.505+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bf53bdf620 con 0x55bf53ea2970 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f9446a00640 1 -- 172.21.15.103:0/693230187 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945800f9a0 con 0x55bf53ea2970 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f9446a00640 1 -- 172.21.15.103:0/693230187 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f94540cdce0 con 0x55bf53ea2970 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f946aea5740 10 client.25757 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f946aea5740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f946aea5740 20 client.25757 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f946aea5740 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53eb7830 0x55bf53eb9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f946aea5740 4 client.25757 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:26:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.506+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf53e17600 con 0x55bf53eb7830 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.507+0000 7f9464c00640 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53eb7830 0x55bf53eb9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.508+0000 7f9446a00640 10 client.25757 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.508+0000 7f9464c00640 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53eb7830 0x55bf53eb9cb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe3b021b2f5d248 server_cookie=495b5944f3176eca in_seq=0 out_seq=0 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f9446a00640 1 -- 172.21.15.103:0/693230187 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf53e17600 con 0x55bf53eb7830 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f9446a00640 10 client.25757 handle_command_reply: tid=0 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 client.25757 shutdown 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 msgr2=0x55bf53eabfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53eabfe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94540cc760 tx=0x7f94540cc3b0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 shutdown_connections 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53eb7830 0x55bf53eb9cb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53ead250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea52a0 0x55bf53eac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 --2- 172.21.15.103:0/693230187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea2970 0x55bf53eabfe0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f9446a00640 0 client.25757 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 >> 172.21.15.103:0/693230187 conn(0x55bf53e9dac0 msgr2=0x55bf53e9cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 shutdown_connections 2024-09-11T00:26:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 1 -- 172.21.15.103:0/693230187 wait complete. 2024-09-11T00:26:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.509+0000 7f946aea5740 20 client.25757 trim_cache size 0 max 16384 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 Processor -- start 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 -- start start 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea2970 0x55bf53ea1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53ea2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea7ce0 0x55bf53e9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94601b0050 con 0x55bf53ea7ce0 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ad860 con 0x55bf53ea52a0 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f946aea5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ab6b0 con 0x55bf53ea2970 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53ea2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.529 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9465600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea2970 0x55bf53ea1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53ea2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53854/0 (socket says 172.21.15.103:53854) 2024-09-11T00:26:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 -- 172.21.15.103:0/3251764587 learned_addr learned my addr 172.21.15.103:0/3251764587 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9465600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea2970 0x55bf53ea1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:33192/0 (socket says 172.21.15.103:33192) 2024-09-11T00:26:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9466000640 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea7ce0 0x55bf53e9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 -- 172.21.15.103:0/3251764587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea2970 msgr2=0x55bf53ea1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea2970 0x55bf53ea1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 -- 172.21.15.103:0/3251764587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea7ce0 msgr2=0x55bf53e9f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea7ce0 0x55bf53e9f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.527+0000 7f9464c00640 1 -- 172.21.15.103:0/3251764587 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf53ea50d0 con 0x55bf53ea52a0 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.528+0000 7f9464c00640 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53ea2280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55bf53eba4f0 tx=0x7f944c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4e9a3c5da95ee7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.528+0000 7f9451c00640 1 -- 172.21.15.103:0/3251764587 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c020040 con 0x55bf53ea52a0 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.528+0000 7f9451c00640 1 -- 172.21.15.103:0/3251764587 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944c0054d0 con 0x55bf53ea52a0 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.528+0000 7f9451c00640 1 -- 172.21.15.103:0/3251764587 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c004e20 con 0x55bf53ea52a0 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 -- 172.21.15.103:0/3251764587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 msgr2=0x55bf53ea2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53ea2280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55bf53eba4f0 tx=0x7f944c005f90 comp rx=0 tx=0).stop 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 -- 172.21.15.103:0/3251764587 shutdown_connections 2024-09-11T00:26:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea7ce0 0x55bf53e9f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53ea2280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 --2- 172.21.15.103:0/3251764587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea2970 0x55bf53ea1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 -- 172.21.15.103:0/3251764587 >> 172.21.15.103:0/3251764587 conn(0x55bf53e9dac0 msgr2=0x55bf53ea80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 -- 172.21.15.103:0/3251764587 shutdown_connections 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 -- 172.21.15.103:0/3251764587 wait complete. 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 -- start start 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.529+0000 7f946aea5740 1 Processor -- start 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f946aea5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea2970 0x55bf53eb2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f946aea5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53eb3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f946aea5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eb3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f946aea5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94601abd20 con 0x55bf53ea2970 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f946aea5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ad190 con 0x55bf53ea52a0 2024-09-11T00:26:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f946aea5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94601ac3f0 con 0x55bf53ea7ce0 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f946aea5740 10 client.0 _ll_register_callbacks cb 0x7f94540c9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9466000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eb3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9466000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eb3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:33206/0 (socket says 172.21.15.103:33206) 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9466000640 1 -- 172.21.15.103:0/324002918 learned_addr learned my addr 172.21.15.103:0/324002918 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9465600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea2970 0x55bf53eb2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9464c00640 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53eb3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9466000640 1 -- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 msgr2=0x55bf53eb3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9466000640 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53eb3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.530+0000 7f9466000640 1 -- 172.21.15.103:0/324002918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea2970 msgr2=0x55bf53eb2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9466000640 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea2970 0x55bf53eb2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9466000640 1 -- 172.21.15.103:0/324002918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf53eb18e0 con 0x55bf53ea7ce0 2024-09-11T00:26:49.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9465600640 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea2970 0x55bf53eb2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:26:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9466000640 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eb3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9458019000 tx=0x7f9458004ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0294f580fc1031 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:26:49.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9446a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:26:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9446a00640 1 -- 172.21.15.103:0/324002918 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945801e070 con 0x55bf53ea7ce0 2024-09-11T00:26:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9446a00640 1 -- 172.21.15.103:0/324002918 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f94580043a0 con 0x55bf53ea7ce0 2024-09-11T00:26:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944c0054d0 con 0x55bf53ea7ce0 2024-09-11T00:26:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.531+0000 7f9446a00640 1 -- 172.21.15.103:0/324002918 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9458041050 con 0x55bf53ea7ce0 2024-09-11T00:26:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 10 client.25801 fetch_fsmap learned FSMap version 35 2024-09-11T00:26:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bf53f2d450 con 0x55bf53ea7ce0 2024-09-11T00:26:49.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f9446a00640 1 -- 172.21.15.103:0/324002918 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944c0054d0 con 0x55bf53ea7ce0 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f9446a00640 1 -- 172.21.15.103:0/324002918 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f9458018070 con 0x55bf53ea7ce0 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 10 client.25801 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 20 client.25801 populate_metadata read hostname 'smithi103' 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53f2dce0 0x55bf53f30100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 4 client.25801 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf53e130b0 con 0x55bf53f2dce0 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.532+0000 7f9465600640 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53f2dce0 0x55bf53f30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.533+0000 7f9465600640 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53f2dce0 0x55bf53f30100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6629aebadcb5b16 server_cookie=b79927c7f1a3a154 in_seq=0 out_seq=0 2024-09-11T00:26:49.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.533+0000 7f9446a00640 10 client.25801 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f9446a00640 1 -- 172.21.15.103:0/324002918 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf53e130b0 con 0x55bf53f2dce0 2024-09-11T00:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f9446a00640 10 client.25801 handle_command_reply: tid=0 2024-09-11T00:26:49.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 client.25801 shutdown 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 msgr2=0x55bf53eb3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eb3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9458019000 tx=0x7f9458004ec0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 shutdown_connections 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55bf53f2dce0 0x55bf53f30100 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bf53ea7ce0 0x55bf53eb3690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55bf53ea52a0 0x55bf53eb3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f9446a00640 0 client.25801 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 --2- 172.21.15.103:0/324002918 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bf53ea2970 0x55bf53eb2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.548+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 >> 172.21.15.103:0/324002918 conn(0x55bf53e9dac0 msgr2=0x55bf53ea1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.549+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 shutdown_connections 2024-09-11T00:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.549+0000 7f946aea5740 1 -- 172.21.15.103:0/324002918 wait complete. 2024-09-11T00:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.549+0000 7f946aea5740 20 client.25801 trim_cache size 0 max 16384 2024-09-11T00:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:26:49.552 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f94480b53e0 msgr2=0x7f94480b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f94480b53e0 0x7f94480b78a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9458011760 tx=0x7f9458000f40 comp rx=0 tx=0).stop 2024-09-11T00:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f946006b470 msgr2=0x7f946019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f946006b470 0x7f946019add0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9454005940 tx=0x7f94540042f0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 shutdown_connections 2024-09-11T00:26:49.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f94480b53e0 0x7f94480b78a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94601084a0 0x7f946019c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94601058f0 0x7f946019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 --2- 172.21.15.103:0/1168262664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f946006b470 0x7f946019add0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:26:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 >> 172.21.15.103:0/1168262664 conn(0x7f94600fcf30 msgr2=0x7f94600ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:26:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 shutdown_connections 2024-09-11T00:26:49.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:26:49.550+0000 7f9466a00640 1 -- 172.21.15.103:0/1168262664 wait complete. 2024-09-11T00:26:50.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:50 smithi103 ceph-mon[30578]: pgmap v511: 129 pgs: 129 active+clean; 57 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 408 op/s 2024-09-11T00:26:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:50 smithi117 ceph-mon[35525]: pgmap v511: 129 pgs: 129 active+clean; 57 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 408 op/s 2024-09-11T00:26:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:50 smithi183 ceph-mon[35934]: pgmap v511: 129 pgs: 129 active+clean; 57 MiB data, 767 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 408 op/s 2024-09-11T00:26:52.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:52 smithi103 ceph-mon[30578]: pgmap v512: 129 pgs: 129 active+clean; 59 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 600 op/s 2024-09-11T00:26:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:52 smithi117 ceph-mon[35525]: pgmap v512: 129 pgs: 129 active+clean; 59 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 600 op/s 2024-09-11T00:26:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:52 smithi183 ceph-mon[35934]: pgmap v512: 129 pgs: 129 active+clean; 59 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 600 op/s 2024-09-11T00:26:54.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:54 smithi103 ceph-mon[30578]: pgmap v513: 129 pgs: 129 active+clean; 59 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 417 op/s 2024-09-11T00:26:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:54 smithi117 ceph-mon[35525]: pgmap v513: 129 pgs: 129 active+clean; 59 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 417 op/s 2024-09-11T00:26:54.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:54 smithi183 ceph-mon[35934]: pgmap v513: 129 pgs: 129 active+clean; 59 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 417 op/s 2024-09-11T00:26:56.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:56 smithi103 ceph-mon[30578]: pgmap v514: 129 pgs: 129 active+clean; 56 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 599 op/s 2024-09-11T00:26:56.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:56 smithi117 ceph-mon[35525]: pgmap v514: 129 pgs: 129 active+clean; 56 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 599 op/s 2024-09-11T00:26:56.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:56 smithi183 ceph-mon[35934]: pgmap v514: 129 pgs: 129 active+clean; 56 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 599 op/s 2024-09-11T00:26:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:26:58 smithi103 ceph-mon[30578]: pgmap v515: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-11T00:26:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:26:58 smithi117 ceph-mon[35525]: pgmap v515: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-11T00:26:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:26:58 smithi183 ceph-mon[35934]: pgmap v515: 129 pgs: 129 active+clean; 56 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-11T00:27:00.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:00 smithi103 ceph-mon[30578]: pgmap v516: 129 pgs: 129 active+clean; 57 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T00:27:00.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:00 smithi117 ceph-mon[35525]: pgmap v516: 129 pgs: 129 active+clean; 57 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T00:27:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:00 smithi183 ceph-mon[35934]: pgmap v516: 129 pgs: 129 active+clean; 57 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T00:27:02.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:02 smithi103 ceph-mon[30578]: pgmap v517: 129 pgs: 129 active+clean; 57 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 544 op/s 2024-09-11T00:27:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 -- 172.21.15.103:0/2337021390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 msgr2=0x7f45fc102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 --2- 172.21.15.103:0/2337021390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 0x7f45fc102b90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009cd0 tx=0x7f45ec02a8f0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 -- 172.21.15.103:0/2337021390 shutdown_connections 2024-09-11T00:27:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 --2- 172.21.15.103:0/2337021390 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45fc1042a0 0x7f45fc10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 --2- 172.21.15.103:0/2337021390 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45fc1038e0 0x7f45fc103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 --2- 172.21.15.103:0/2337021390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 0x7f45fc102b90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 -- 172.21.15.103:0/2337021390 >> 172.21.15.103:0/2337021390 conn(0x7f45fc0fe030 msgr2=0x7f45fc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 -- 172.21.15.103:0/2337021390 shutdown_connections 2024-09-11T00:27:02.685 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.682+0000 7f4602600640 1 -- 172.21.15.103:0/2337021390 wait complete. 2024-09-11T00:27:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.683+0000 7f4602600640 1 Processor -- start 2024-09-11T00:27:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.683+0000 7f4602600640 1 -- start start 2024-09-11T00:27:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4602600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 0x7f45fc1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4602600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45fc1038e0 0x7f45fc197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4602600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45fc1042a0 0x7f45fc197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4602600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10c280 con 0x7f45fc1042a0 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4602600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10c100 con 0x7f45fc1038e0 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4602600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc10c400 con 0x7f45fc102790 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4601200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 0x7f45fc1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45fc1038e0 0x7f45fc197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4601200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 0x7f45fc1969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:48422/0 (socket says 172.21.15.103:48422) 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4601200640 1 -- 172.21.15.103:0/3842191616 learned_addr learned my addr 172.21.15.103:0/3842191616 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f4601c00640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45fc1042a0 0x7f45fc197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f45fbe00640 1 -- 172.21.15.103:0/3842191616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 msgr2=0x7f45fc1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f45fbe00640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 0x7f45fc1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f45fbe00640 1 -- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45fc1042a0 msgr2=0x7f45fc197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f45fbe00640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45fc1042a0 0x7f45fc197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.684+0000 7f45fbe00640 1 -- 172.21.15.103:0/3842191616 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc198500 con 0x7f45fc1038e0 2024-09-11T00:27:02.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:02 smithi117 ceph-mon[35525]: pgmap v517: 129 pgs: 129 active+clean; 57 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 544 op/s 2024-09-11T00:27:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.685+0000 7f45fbe00640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45fc1038e0 0x7f45fc197010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e9f0 tx=0x7f45e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.685+0000 7f45f9600640 1 -- 172.21.15.103:0/3842191616 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e800ccc0 con 0x7f45fc1038e0 2024-09-11T00:27:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.685+0000 7f45f9600640 1 -- 172.21.15.103:0/3842191616 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45e80108e0 con 0x7f45fc1038e0 2024-09-11T00:27:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.685+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1987f0 con 0x7f45fc1038e0 2024-09-11T00:27:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.685+0000 7f45f9600640 1 -- 172.21.15.103:0/3842191616 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8018880 con 0x7f45fc1038e0 2024-09-11T00:27:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.685+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc19eec0 con 0x7f45fc1038e0 2024-09-11T00:27:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.688+0000 7f4606a20740 1 Processor -- start 2024-09-11T00:27:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4606a20740 1 -- start start 2024-09-11T00:27:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4606a20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2720970 0x55d1e2722d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4606a20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e2725690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4606a20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27281a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4606a20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1e26936b0 con 0x55d1e27232a0 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4606a20740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2682ae0 con 0x55d1e2720970 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4606a20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1e26868f0 con 0x55d1e2725ce0 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4601c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2720970 0x55d1e2722d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4601200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e2725690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4601c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2720970 0x55d1e2722d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:56726/0 (socket says 172.21.15.103:56726) 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.689+0000 7f4601c00640 1 -- 172.21.15.103:0/2087463580 learned_addr learned my addr 172.21.15.103:0/2087463580 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f4601c00640 1 -- 172.21.15.103:0/2087463580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 msgr2=0x55d1e27281a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:27:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f4601c00640 1 --2- 172.21.15.103:0/2087463580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27281a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f4601c00640 1 -- 172.21.15.103:0/2087463580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 msgr2=0x55d1e2725690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f4601c00640 1 --2- 172.21.15.103:0/2087463580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e2725690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f4601c00640 1 -- 172.21.15.103:0/2087463580 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e264f780 con 0x55d1e2720970 2024-09-11T00:27:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f45f9600640 1 -- 172.21.15.103:0/3842191616 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f45e8010430 con 0x7f45fc1038e0 2024-09-11T00:27:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f4601c00640 1 --2- 172.21.15.103:0/2087463580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2720970 0x55d1e2722d60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45f4015260 tx=0x7f45f4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfbf26f23e322b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f45f2000640 1 -- 172.21.15.103:0/2087463580 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f401d070 con 0x55d1e2720970 2024-09-11T00:27:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f45f2000640 1 -- 172.21.15.103:0/2087463580 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f400ba80 con 0x55d1e2720970 2024-09-11T00:27:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.690+0000 7f45f2000640 1 -- 172.21.15.103:0/2087463580 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f40198e0 con 0x55d1e2720970 2024-09-11T00:27:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 -- 172.21.15.103:0/2087463580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2720970 msgr2=0x55d1e2722d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 --2- 172.21.15.103:0/2087463580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2720970 0x55d1e2722d60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45f4015260 tx=0x7f45f4005f90 comp rx=0 tx=0).stop 2024-09-11T00:27:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 -- 172.21.15.103:0/2087463580 shutdown_connections 2024-09-11T00:27:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 --2- 172.21.15.103:0/2087463580 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27281a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 --2- 172.21.15.103:0/2087463580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e2725690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 --2- 172.21.15.103:0/2087463580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2720970 0x55d1e2722d60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 -- 172.21.15.103:0/2087463580 >> 172.21.15.103:0/2087463580 conn(0x55d1e271bac0 msgr2=0x55d1e271c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 -- 172.21.15.103:0/2087463580 shutdown_connections 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f4606a20740 1 -- 172.21.15.103:0/2087463580 wait complete. 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.691+0000 7f45f9600640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f45e00b5270 0x7f45e00b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f45f9600640 1 -- 172.21.15.103:0/3842191616 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f45e80e6270 con 0x7f45fc1038e0 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4601200640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f45e00b5270 0x7f45e00b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 -- start start 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 Processor -- start 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4601200640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f45e00b5270 0x7f45e00b7730 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f45ec031040 tx=0x7f45ec009630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e272a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e272a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e272b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d1e25baab0 con 0x55d1e2720970 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2629570 con 0x55d1e27232a0 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.692+0000 7f4606a20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1e2728cc0 con 0x55d1e2725ce0 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f4606a20740 10 client.0 _ll_register_callbacks cb 0x55d1e2638100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e272a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e272a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:56728/0 (socket says 172.21.15.103:56728) 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 -- 172.21.15.103:0/3060350770 learned_addr learned my addr 172.21.15.103:0/3060350770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f4601c00640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e272b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f4601200640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e272a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 -- 172.21.15.103:0/3060350770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 msgr2=0x55d1e272b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e272b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 -- 172.21.15.103:0/3060350770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 msgr2=0x55d1e272a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e272a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 -- 172.21.15.103:0/3060350770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e23e4790 con 0x55d1e27232a0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f4601c00640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e272b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45fbe00640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e272a740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ac680 tx=0x7f45e80ac100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed096e5bd836bfdf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.693+0000 7f45e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f45e6a00640 1 -- 172.21.15.103:0/3060350770 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e80ac6f0 con 0x55d1e27232a0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f45e6a00640 1 -- 172.21.15.103:0/3060350770 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f45e8010a90 con 0x55d1e27232a0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f400ba80 con 0x55d1e27232a0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f45e6a00640 1 -- 172.21.15.103:0/3060350770 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f45e80b1120 con 0x55d1e27232a0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 10 client.25775 fetch_fsmap learned FSMap version 35 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d1e245d620 con 0x55d1e27232a0 2024-09-11T00:27:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f45e6a00640 1 -- 172.21.15.103:0/3060350770 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f400ba80 con 0x55d1e27232a0 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f45e6a00640 1 -- 172.21.15.103:0/3060350770 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f45e80ab430 con 0x55d1e27232a0 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 10 client.25775 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 20 client.25775 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27359d0 0x55d1e2737df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 4 client.25775 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.694+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e2695600 con 0x55d1e27359d0 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.695+0000 7f4601200640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27359d0 0x55d1e2737df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.695+0000 7f4601200640 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27359d0 0x55d1e2737df0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a84649a69c08103 server_cookie=c51e1ea3e465e071 in_seq=0 out_seq=0 2024-09-11T00:27:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.695+0000 7f45e6a00640 10 client.25775 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f45e6a00640 1 -- 172.21.15.103:0/3060350770 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1e2695600 con 0x55d1e27359d0 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f45e6a00640 10 client.25775 handle_command_reply: tid=0 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 client.25775 shutdown 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 msgr2=0x55d1e272a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e272a740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ac680 tx=0x7f45e80ac100 comp rx=0 tx=0).stop 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 shutdown_connections 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27359d0 0x55d1e2737df0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e272b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e272a740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f45e6a00640 0 client.25775 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 --2- 172.21.15.103:0/3060350770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e272a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 >> 172.21.15.103:0/3060350770 conn(0x55d1e271bac0 msgr2=0x55d1e271abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:02.703 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.696+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 shutdown_connections 2024-09-11T00:27:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.697+0000 7f4606a20740 1 -- 172.21.15.103:0/3060350770 wait complete. 2024-09-11T00:27:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.697+0000 7f4606a20740 20 client.25775 trim_cache size 0 max 16384 2024-09-11T00:27:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.716+0000 7f4606a20740 1 Processor -- start 2024-09-11T00:27:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.716+0000 7f4606a20740 1 -- start start 2024-09-11T00:27:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4606a20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2720970 0x55d1e2716ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4606a20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e271eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4606a20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2725ce0 0x55d1e271f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4606a20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1abbb0 con 0x55d1e27232a0 2024-09-11T00:27:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4606a20740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1a93c0 con 0x55d1e2725ce0 2024-09-11T00:27:02.719 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4606a20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1a7210 con 0x55d1e2720970 2024-09-11T00:27:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4601200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2720970 0x55d1e2716ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4601200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2720970 0x55d1e2716ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:48440/0 (socket says 172.21.15.103:48440) 2024-09-11T00:27:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4601200640 1 -- 172.21.15.103:0/1134739297 learned_addr learned my addr 172.21.15.103:0/1134739297 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f4601c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2725ce0 0x55d1e271f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.720 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.717+0000 7f45fbe00640 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e271eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f4601c00640 1 -- 172.21.15.103:0/1134739297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2720970 msgr2=0x55d1e2716ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f4601c00640 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2720970 0x55d1e2716ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f4601c00640 1 -- 172.21.15.103:0/1134739297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 msgr2=0x55d1e271eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f4601c00640 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e271eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f4601c00640 1 -- 172.21.15.103:0/1134739297 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e27230d0 con 0x55d1e2725ce0 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f45fbe00640 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e271eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f4601c00640 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2725ce0 0x55d1e271f050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f45f4005890 tx=0x7f45f4015fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946aadddfdd13ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f45f1600640 1 -- 172.21.15.103:0/1134739297 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f401d040 con 0x55d1e2725ce0 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f45f1600640 1 -- 172.21.15.103:0/1134739297 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f400d2e0 con 0x55d1e2725ce0 2024-09-11T00:27:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.718+0000 7f45f1600640 1 -- 172.21.15.103:0/1134739297 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4004bc0 con 0x55d1e2725ce0 2024-09-11T00:27:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.719+0000 7f4606a20740 1 -- 172.21.15.103:0/1134739297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2725ce0 msgr2=0x55d1e271f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.724 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.719+0000 7f4606a20740 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2725ce0 0x55d1e271f050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f45f4005890 tx=0x7f45f4015fd0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.719+0000 7f4606a20740 1 -- 172.21.15.103:0/1134739297 shutdown_connections 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.719+0000 7f4606a20740 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e2725ce0 0x55d1e271f050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.719+0000 7f4606a20740 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e27232a0 0x55d1e271eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.719+0000 7f4606a20740 1 --2- 172.21.15.103:0/1134739297 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2720970 0x55d1e2716ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.719+0000 7f4606a20740 1 -- 172.21.15.103:0/1134739297 >> 172.21.15.103:0/1134739297 conn(0x55d1e271bac0 msgr2=0x55d1e271fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 -- 172.21.15.103:0/1134739297 shutdown_connections 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 -- 172.21.15.103:0/1134739297 wait complete. 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 -- start start 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 Processor -- start 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e2730d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e2731260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.725 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27317a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1a7880 con 0x55d1e2720970 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1a8cf0 con 0x55d1e27232a0 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.720+0000 7f4606a20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1a7f50 con 0x55d1e2725ce0 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4606a20740 10 client.0 _ll_register_callbacks cb 0x7f45e800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e2731260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f45fbe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e2731260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:56736/0 (socket says 172.21.15.103:56736) 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f45fbe00640 1 -- 172.21.15.103:0/4242944040 learned_addr learned my addr 172.21.15.103:0/4242944040 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601200640 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e2730d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601c00640 1 -- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 msgr2=0x55d1e2731260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601c00640 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e2731260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601c00640 1 -- 172.21.15.103:0/4242944040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 msgr2=0x55d1e2730d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601c00640 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e2730d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601c00640 1 -- 172.21.15.103:0/4242944040 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e272fa40 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f45fbe00640 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e2731260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f4601c00640 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27317a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45f4006010 tx=0x7f45f4004b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a93227de12e8daea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.721+0000 7f45e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f45e6a00640 1 -- 172.21.15.103:0/4242944040 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f401d040 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f45e6a00640 1 -- 172.21.15.103:0/4242944040 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f45f4004030 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f400d2e0 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f45e6a00640 1 -- 172.21.15.103:0/4242944040 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f45f4004270 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f4606a20740 10 client.25816 fetch_fsmap learned FSMap version 35 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d1e27a2df0 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f45e6a00640 1 -- 172.21.15.103:0/4242944040 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45f400d2e0 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f45e6a00640 1 -- 172.21.15.103:0/4242944040 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f45f4003aa0 con 0x55d1e2725ce0 2024-09-11T00:27:02.727 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f4606a20740 10 client.25816 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f4606a20740 10 client.25816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f4606a20740 20 client.25816 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.722+0000 7f4606a20740 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27a3650 0x55d1e27a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.723+0000 7f4606a20740 4 client.25816 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.723+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1e26910b0 con 0x55d1e27a3650 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.723+0000 7f4601200640 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27a3650 0x55d1e27a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.723+0000 7f4601200640 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27a3650 0x55d1e27a5a70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62431e47bfe70434 server_cookie=c286250c5fbf317d in_seq=0 out_seq=0 2024-09-11T00:27:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.723+0000 7f45e6a00640 10 client.25816 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f45e6a00640 1 -- 172.21.15.103:0/4242944040 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1e26910b0 con 0x55d1e27a3650 2024-09-11T00:27:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f45e6a00640 10 client.25816 handle_command_reply: tid=0 2024-09-11T00:27:02.736 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 client.25816 shutdown 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 msgr2=0x55d1e27317a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27317a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45f4006010 tx=0x7f45f4004b30 comp rx=0 tx=0).stop 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 shutdown_connections 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d1e27a3650 0x55d1e27a5a70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1e2725ce0 0x55d1e27317a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d1e27232a0 0x55d1e2731260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 --2- 172.21.15.103:0/4242944040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d1e2720970 0x55d1e2730d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f45e6a00640 0 client.25816 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 >> 172.21.15.103:0/4242944040 conn(0x55d1e271bac0 msgr2=0x55d1e271fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 shutdown_connections 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 1 -- 172.21.15.103:0/4242944040 wait complete. 2024-09-11T00:27:02.737 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.734+0000 7f4606a20740 20 client.25816 trim_cache size 0 max 16384 2024-09-11T00:27:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:27:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:27:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:27:02.738 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f45e00b5270 msgr2=0x7f45e00b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f45e00b5270 0x7f45e00b7730 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f45ec031040 tx=0x7f45ec009630 comp rx=0 tx=0).stop 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45fc1038e0 msgr2=0x7f45fc197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45fc1038e0 0x7f45fc197010 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e9f0 tx=0x7f45e800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 shutdown_connections 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f45e00b5270 0x7f45e00b7730 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45fc1042a0 0x7f45fc197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45fc1038e0 0x7f45fc197010 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 --2- 172.21.15.103:0/3842191616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45fc102790 0x7f45fc1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 >> 172.21.15.103:0/3842191616 conn(0x7f45fc0fe030 msgr2=0x7f45fc106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:02.739 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 shutdown_connections 2024-09-11T00:27:02.740 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:02.735+0000 7f4602600640 1 -- 172.21.15.103:0/3842191616 wait complete. 2024-09-11T00:27:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:02 smithi183 ceph-mon[35934]: pgmap v517: 129 pgs: 129 active+clean; 57 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 544 op/s 2024-09-11T00:27:04.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:04 smithi103 ceph-mon[30578]: pgmap v518: 129 pgs: 129 active+clean; 57 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-09-11T00:27:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:04 smithi117 ceph-mon[35525]: pgmap v518: 129 pgs: 129 active+clean; 57 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-09-11T00:27:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:04 smithi183 ceph-mon[35934]: pgmap v518: 129 pgs: 129 active+clean; 57 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 347 op/s 2024-09-11T00:27:06.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:06 smithi103 ceph-mon[30578]: pgmap v519: 129 pgs: 129 active+clean; 56 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 515 op/s 2024-09-11T00:27:06.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:06 smithi117 ceph-mon[35525]: pgmap v519: 129 pgs: 129 active+clean; 56 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 515 op/s 2024-09-11T00:27:06.703 INFO:tasks.workunit.client.0.smithi103.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-09-11T00:27:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:06 smithi183 ceph-mon[35934]: pgmap v519: 129 pgs: 129 active+clean; 56 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 515 op/s 2024-09-11T00:27:07.281 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xffcdd 2024-09-11T00:27:08.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:08 smithi103 ceph-mon[30578]: pgmap v520: 129 pgs: 129 active+clean; 56 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-09-11T00:27:08.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:08 smithi117 ceph-mon[35525]: pgmap v520: 129 pgs: 129 active+clean; 56 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-09-11T00:27:08.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:08 smithi183 ceph-mon[35934]: pgmap v520: 129 pgs: 129 active+clean; 56 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-09-11T00:27:09.939 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xffd97 2024-09-11T00:27:10.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:10 smithi103 ceph-mon[30578]: pgmap v521: 129 pgs: 129 active+clean; 58 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 387 op/s 2024-09-11T00:27:10.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:10 smithi117 ceph-mon[35525]: pgmap v521: 129 pgs: 129 active+clean; 58 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 387 op/s 2024-09-11T00:27:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:10 smithi183 ceph-mon[35934]: pgmap v521: 129 pgs: 129 active+clean; 58 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 387 op/s 2024-09-11T00:27:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:12 smithi117 ceph-mon[35525]: pgmap v522: 129 pgs: 129 active+clean; 59 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 471 op/s 2024-09-11T00:27:12.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:12 smithi183 ceph-mon[35934]: pgmap v522: 129 pgs: 129 active+clean; 59 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 471 op/s 2024-09-11T00:27:12.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:12 smithi103 ceph-mon[30578]: pgmap v522: 129 pgs: 129 active+clean; 59 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 471 op/s 2024-09-11T00:27:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:14 smithi117 ceph-mon[35525]: pgmap v523: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-09-11T00:27:14.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:14 smithi183 ceph-mon[35934]: pgmap v523: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-09-11T00:27:14.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:14 smithi103 ceph-mon[30578]: pgmap v523: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 328 op/s 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.868+0000 7fb294a00640 1 -- 172.21.15.103:0/1438524244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 msgr2=0x7fb290102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.868+0000 7fb294a00640 1 --2- 172.21.15.103:0/1438524244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290102d70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb280009900 tx=0x7fb28002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.868+0000 7fb294a00640 1 -- 172.21.15.103:0/1438524244 shutdown_connections 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.868+0000 7fb294a00640 1 --2- 172.21.15.103:0/1438524244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb290104480 0x7fb29010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.868+0000 7fb294a00640 1 --2- 172.21.15.103:0/1438524244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb290103ac0 0x7fb290103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.868+0000 7fb294a00640 1 --2- 172.21.15.103:0/1438524244 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290102d70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.868+0000 7fb294a00640 1 -- 172.21.15.103:0/1438524244 >> 172.21.15.103:0/1438524244 conn(0x7fb2900fe250 msgr2=0x7fb290100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.869+0000 7fb294a00640 1 -- 172.21.15.103:0/1438524244 shutdown_connections 2024-09-11T00:27:15.872 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.869+0000 7fb294a00640 1 -- 172.21.15.103:0/1438524244 wait complete. 2024-09-11T00:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 Processor -- start 2024-09-11T00:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 -- start start 2024-09-11T00:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb290103ac0 0x7fb2901993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb290104480 0x7fb290199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb29010e630 con 0x7fb290103ac0 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb29010e4b0 con 0x7fb290102970 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb294a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb29010e7b0 con 0x7fb290104480 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb28de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb28de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53116/0 (socket says 172.21.15.103:53116) 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb28de00640 1 -- 172.21.15.103:0/3304777111 learned_addr learned my addr 172.21.15.103:0/3304777111 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb28e800640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb290104480 0x7fb290199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.870+0000 7fb28d400640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb290103ac0 0x7fb2901993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb28de00640 1 -- 172.21.15.103:0/3304777111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb290104480 msgr2=0x7fb290199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb28de00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb290104480 0x7fb290199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.875 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb28de00640 1 -- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb290103ac0 msgr2=0x7fb2901993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb28de00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb290103ac0 0x7fb2901993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb28de00640 1 -- 172.21.15.103:0/3304777111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29019a890 con 0x7fb290102970 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb28de00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290198d50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb280031490 tx=0x7fb2800045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb286a00640 1 -- 172.21.15.103:0/3304777111 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280035030 con 0x7fb290102970 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb286a00640 1 -- 172.21.15.103:0/3304777111 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb280041070 con 0x7fb290102970 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb286a00640 1 -- 172.21.15.103:0/3304777111 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280031e30 con 0x7fb290102970 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.871+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29019ab20 con 0x7fb290102970 2024-09-11T00:27:15.876 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.872+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb290075380 con 0x7fb290102970 2024-09-11T00:27:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.875+0000 7fb297a41740 1 Processor -- start 2024-09-11T00:27:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.875+0000 7fb286a00640 1 -- 172.21.15.103:0/3304777111 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb2800315c0 con 0x7fb290102970 2024-09-11T00:27:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb297a41740 1 -- start start 2024-09-11T00:27:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb297a41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.878 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb297a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94df0c0 0x55aaa94e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb297a41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb297a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aaa944f6b0 con 0x55aaa94df0c0 2024-09-11T00:27:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb297a41740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aaa943eae0 con 0x55aaa94e1b00 2024-09-11T00:27:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb297a41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aaa94428f0 con 0x55aaa94dc790 2024-09-11T00:27:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb28e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb28e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94deb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:40522/0 (socket says 172.21.15.103:40522) 2024-09-11T00:27:15.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.876+0000 7fb28e800640 1 -- 172.21.15.103:0/3828991597 learned_addr learned my addr 172.21.15.103:0/3828991597 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.877+0000 7fb28de00640 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94df0c0 0x55aaa94e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.877+0000 7fb28e800640 1 -- 172.21.15.103:0/3828991597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 msgr2=0x55aaa94e3fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:27:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.877+0000 7fb28e800640 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e3fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.877+0000 7fb28e800640 1 -- 172.21.15.103:0/3828991597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94df0c0 msgr2=0x55aaa94e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.877+0000 7fb28e800640 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94df0c0 0x55aaa94e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.877+0000 7fb28e800640 1 -- 172.21.15.103:0/3828991597 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa940b780 con 0x55aaa94dc790 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.877+0000 7fb28e800640 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94deb80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb288009f90 tx=0x7fb2880118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36485e1e39158b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.878+0000 7fb286a00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb2740b52e0 0x7fb2740b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.878+0000 7fb27b000640 1 -- 172.21.15.103:0/3828991597 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288004d20 con 0x55aaa94dc790 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.878+0000 7fb27b000640 1 -- 172.21.15.103:0/3828991597 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28801a970 con 0x55aaa94dc790 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.878+0000 7fb27b000640 1 -- 172.21.15.103:0/3828991597 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288004430 con 0x55aaa94dc790 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.878+0000 7fb28de00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb2740b52e0 0x7fb2740b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.878+0000 7fb286a00640 1 -- 172.21.15.103:0/3304777111 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb28010a930 con 0x7fb290102970 2024-09-11T00:27:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 -- 172.21.15.103:0/3828991597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 msgr2=0x55aaa94deb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb28de00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb2740b52e0 0x7fb2740b77a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2800cd330 tx=0x7fb280110550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94deb80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb288009f90 tx=0x7fb2880118d0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 -- 172.21.15.103:0/3828991597 shutdown_connections 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94df0c0 0x55aaa94e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 --2- 172.21.15.103:0/3828991597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94deb80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 -- 172.21.15.103:0/3828991597 >> 172.21.15.103:0/3828991597 conn(0x55aaa94d78e0 msgr2=0x55aaa94d8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 -- 172.21.15.103:0/3828991597 shutdown_connections 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 -- 172.21.15.103:0/3828991597 wait complete. 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.879+0000 7fb297a41740 1 -- start start 2024-09-11T00:27:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 1 Processor -- start 2024-09-11T00:27:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94dc790 0x55aaa94e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94df0c0 0x55aaa94e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aaa9376ab0 con 0x55aaa94dc790 2024-09-11T00:27:15.887 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55aaa93e5570 con 0x55aaa94e1b00 2024-09-11T00:27:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aaa94e4ae0 con 0x55aaa94df0c0 2024-09-11T00:27:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb297a41740 10 client.0 _ll_register_callbacks cb 0x55aaa93f4100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb28d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94df0c0 0x55aaa94e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb28e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94dc790 0x55aaa94e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb28d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94df0c0 0x55aaa94e6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:40536/0 (socket says 172.21.15.103:40536) 2024-09-11T00:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb28d400640 1 -- 172.21.15.103:0/46388404 learned_addr learned my addr 172.21.15.103:0/46388404 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.880+0000 7fb28e800640 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb28d400640 1 -- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 msgr2=0x55aaa94e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb28d400640 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.889 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb28d400640 1 -- 172.21.15.103:0/46388404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94dc790 msgr2=0x55aaa94e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb28d400640 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94dc790 0x55aaa94e5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb28d400640 1 -- 172.21.15.103:0/46388404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa91a0790 con 0x55aaa94df0c0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb28e800640 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb28d400640 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94df0c0 0x55aaa94e6560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb27c011690 tx=0x7fb27c011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed3f3ae3ef8d858 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb26f400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb26f400640 1 -- 172.21.15.103:0/46388404 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c01f060 con 0x55aaa94df0c0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb28801a970 con 0x55aaa94df0c0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.881+0000 7fb26f400640 1 -- 172.21.15.103:0/46388404 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb27c00fac0 con 0x55aaa94df0c0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.882+0000 7fb26f400640 1 -- 172.21.15.103:0/46388404 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb27c026050 con 0x55aaa94df0c0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.882+0000 7fb297a41740 10 client.25828 fetch_fsmap learned FSMap version 35 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.882+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aaa9219620 con 0x55aaa94df0c0 2024-09-11T00:27:15.890 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.882+0000 7fb26f400640 1 -- 172.21.15.103:0/46388404 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb28801a970 con 0x55aaa94df0c0 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.882+0000 7fb26f400640 1 -- 172.21.15.103:0/46388404 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fb27c018340 con 0x55aaa94df0c0 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb297a41740 10 client.25828 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb297a41740 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb297a41740 20 client.25828 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb297a41740 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa94f1870 0x55aaa94f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb297a41740 4 client.25828 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaa9451600 con 0x55aaa94f1870 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb28e800640 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa94f1870 0x55aaa94f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb28e800640 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa94f1870 0x55aaa94f3ca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b50185da19fe550 server_cookie=f6bf95a65cf944de in_seq=0 out_seq=0 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.883+0000 7fb26f400640 10 client.25828 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.884+0000 7fb26f400640 1 -- 172.21.15.103:0/46388404 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aaa9451600 con 0x55aaa94f1870 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.884+0000 7fb26f400640 10 client.25828 handle_command_reply: tid=0 2024-09-11T00:27:15.891 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.884+0000 7fb297a41740 1 client.25828 shutdown 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.884+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94df0c0 msgr2=0x55aaa94e6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.884+0000 7fb297a41740 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94df0c0 0x55aaa94e6560 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb27c011690 tx=0x7fb27c011b50 comp rx=0 tx=0).stop 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 shutdown_connections 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94e1b00 0x55aaa94e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa94f1870 0x55aaa94f3ca0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94df0c0 0x55aaa94e6560 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 --2- 172.21.15.103:0/46388404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94dc790 0x55aaa94e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb26f400640 0 client.25828 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 >> 172.21.15.103:0/46388404 conn(0x55aaa94d78e0 msgr2=0x55aaa94d6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 shutdown_connections 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 1 -- 172.21.15.103:0/46388404 wait complete. 2024-09-11T00:27:15.892 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.885+0000 7fb297a41740 20 client.25828 trim_cache size 0 max 16384 2024-09-11T00:27:15.906 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.903+0000 7fb297a41740 1 Processor -- start 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.903+0000 7fb297a41740 1 -- start start 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.903+0000 7fb297a41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.903+0000 7fb297a41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.903+0000 7fb297a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94e2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.903+0000 7fb297a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901abcb0 con 0x55aaa94e1b00 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb297a41740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901a94c0 con 0x55aaa94df0c0 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb297a41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901a7310 con 0x55aaa94dc790 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94e1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53144/0 (socket says 172.21.15.103:53144) 2024-09-11T00:27:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28e800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 -- 172.21.15.103:0/2421640391 learned_addr learned my addr 172.21.15.103:0/2421640391 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28e800640 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 -- 172.21.15.103:0/2421640391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 msgr2=0x55aaa94d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 -- 172.21.15.103:0/2421640391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 msgr2=0x55aaa94e2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94e2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.908 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 -- 172.21.15.103:0/2421640391 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa94deef0 con 0x55aaa94df0c0 2024-09-11T00:27:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.904+0000 7fb28d400640 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94e1ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009da0 tx=0x7fb27c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928a8d05c2393798 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.905+0000 7fb27a600640 1 -- 172.21.15.103:0/2421640391 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c011e90 con 0x55aaa94df0c0 2024-09-11T00:27:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.905+0000 7fb27a600640 1 -- 172.21.15.103:0/2421640391 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27c005570 con 0x55aaa94df0c0 2024-09-11T00:27:15.909 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.905+0000 7fb27a600640 1 -- 172.21.15.103:0/2421640391 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00fb20 con 0x55aaa94df0c0 2024-09-11T00:27:15.912 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.905+0000 7fb297a41740 1 -- 172.21.15.103:0/2421640391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 msgr2=0x55aaa94e1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94e1ee0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009da0 tx=0x7fb27c00f730 comp rx=0 tx=0).stop 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 -- 172.21.15.103:0/2421640391 shutdown_connections 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94e2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94e1ee0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 --2- 172.21.15.103:0/2421640391 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 -- 172.21.15.103:0/2421640391 >> 172.21.15.103:0/2421640391 conn(0x55aaa94d78e0 msgr2=0x55aaa94d82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 -- 172.21.15.103:0/2421640391 shutdown_connections 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 -- 172.21.15.103:0/2421640391 wait complete. 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 -- start start 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.906+0000 7fb297a41740 1 Processor -- start 2024-09-11T00:27:15.913 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb297a41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94eda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb297a41740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94ec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb297a41740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94ecda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb297a41740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901a7980 con 0x55aaa94e1b00 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb297a41740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901a8df0 con 0x55aaa94df0c0 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb297a41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2901a8050 con 0x55aaa94dc790 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb297a41740 10 client.0 _ll_register_callbacks cb 0x7fb27c009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94ec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94ec790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53148/0 (socket says 172.21.15.103:53148) 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 -- 172.21.15.103:0/748246245 learned_addr learned my addr 172.21.15.103:0/748246245 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28e800640 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28e800640 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94ecda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 -- 172.21.15.103:0/748246245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 msgr2=0x55aaa94eda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:15.914 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94eda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 -- 172.21.15.103:0/748246245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 msgr2=0x55aaa94ecda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94ecda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.907+0000 7fb28d400640 1 -- 172.21.15.103:0/748246245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa94ed5d0 con 0x55aaa94df0c0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb28d400640 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94ec790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb27c005d00 tx=0x7fb27c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30a76e3c3dcf8832 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb26f400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb26f400640 1 -- 172.21.15.103:0/748246245 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c01f070 con 0x55aaa94df0c0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb27c005570 con 0x55aaa94df0c0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb26f400640 1 -- 172.21.15.103:0/748246245 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb27c005110 con 0x55aaa94df0c0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb26f400640 1 -- 172.21.15.103:0/748246245 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb27c005310 con 0x55aaa94df0c0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb297a41740 10 client.25796 fetch_fsmap learned FSMap version 35 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55aaa955e8d0 con 0x55aaa94df0c0 2024-09-11T00:27:15.915 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.908+0000 7fb26f400640 1 -- 172.21.15.103:0/748246245 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb27c005570 con 0x55aaa94df0c0 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb26f400640 1 -- 172.21.15.103:0/748246245 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7fb27c0038c0 con 0x55aaa94df0c0 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb297a41740 10 client.25796 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb297a41740 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb297a41740 20 client.25796 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb297a41740 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa955f2d0 0x55aaa95616f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb297a41740 4 client.25796 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aaa944d0b0 con 0x55aaa955f2d0 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb28e800640 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa955f2d0 0x55aaa95616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb26f400640 10 client.25796 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:15.916 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.909+0000 7fb28e800640 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa955f2d0 0x55aaa95616f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8aa2aaa552a28a server_cookie=d95b32237f081834 in_seq=0 out_seq=0 2024-09-11T00:27:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb26f400640 1 -- 172.21.15.103:0/748246245 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aaa944d0b0 con 0x55aaa955f2d0 2024-09-11T00:27:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb26f400640 10 client.25796 handle_command_reply: tid=0 2024-09-11T00:27:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 client.25796 shutdown 2024-09-11T00:27:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 msgr2=0x55aaa94ec790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94ec790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb27c005d00 tx=0x7fb27c009fd0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 shutdown_connections 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aaa94e1b00 0x55aaa94ecda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55aaa94df0c0 0x55aaa94ec790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55aaa955f2d0 0x55aaa95616f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 --2- 172.21.15.103:0/748246245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaa94dc790 0x55aaa94eda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb26f400640 0 client.25796 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.922+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 >> 172.21.15.103:0/748246245 conn(0x55aaa94d78e0 msgr2=0x55aaa94e5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.923+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 shutdown_connections 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.923+0000 7fb297a41740 1 -- 172.21.15.103:0/748246245 wait complete. 2024-09-11T00:27:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.923+0000 7fb297a41740 20 client.25796 trim_cache size 0 max 16384 2024-09-11T00:27:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:27:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:27:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:27:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb2740b52e0 msgr2=0x7fb2740b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb2740b52e0 0x7fb2740b77a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb2800cd330 tx=0x7fb280110550 comp rx=0 tx=0).stop 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 msgr2=0x7fb290198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290198d50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb280031490 tx=0x7fb2800045e0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 shutdown_connections 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb2740b52e0 0x7fb2740b77a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb290104480 0x7fb290199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb290103ac0 0x7fb2901993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 --2- 172.21.15.103:0/3304777111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb290102970 0x7fb290198d50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 >> 172.21.15.103:0/3304777111 conn(0x7fb2900fe250 msgr2=0x7fb290106e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 shutdown_connections 2024-09-11T00:27:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:15.924+0000 7fb294a00640 1 -- 172.21.15.103:0/3304777111 wait complete. 2024-09-11T00:27:16.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:16 smithi117 ceph-mon[35525]: pgmap v524: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 467 op/s 2024-09-11T00:27:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:16 smithi183 ceph-mon[35934]: pgmap v524: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 467 op/s 2024-09-11T00:27:16.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:16 smithi103 ceph-mon[30578]: pgmap v524: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 467 op/s 2024-09-11T00:27:18.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:18 smithi117 ceph-mon[35525]: pgmap v525: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2024-09-11T00:27:18.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:18 smithi183 ceph-mon[35934]: pgmap v525: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2024-09-11T00:27:18.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:18 smithi103 ceph-mon[30578]: pgmap v525: 129 pgs: 129 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 299 op/s 2024-09-11T00:27:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:20 smithi117 ceph-mon[35525]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 349 op/s 2024-09-11T00:27:20.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:20 smithi183 ceph-mon[35934]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 349 op/s 2024-09-11T00:27:20.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:20 smithi103 ceph-mon[30578]: pgmap v526: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 349 op/s 2024-09-11T00:27:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:22 smithi117 ceph-mon[35525]: pgmap v527: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 390 op/s 2024-09-11T00:27:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:22 smithi183 ceph-mon[35934]: pgmap v527: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 390 op/s 2024-09-11T00:27:22.832 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0xffdb8 2024-09-11T00:27:22.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:22 smithi103 ceph-mon[30578]: pgmap v527: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 390 op/s 2024-09-11T00:27:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:24 smithi117 ceph-mon[35525]: pgmap v528: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 289 op/s 2024-09-11T00:27:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:24 smithi183 ceph-mon[35934]: pgmap v528: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 289 op/s 2024-09-11T00:27:24.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:24 smithi103 ceph-mon[30578]: pgmap v528: 129 pgs: 129 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 289 op/s 2024-09-11T00:27:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:26 smithi183 ceph-mon[35934]: pgmap v529: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 430 op/s 2024-09-11T00:27:26.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:26 smithi103 ceph-mon[30578]: pgmap v529: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 430 op/s 2024-09-11T00:27:26.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:26 smithi117 ceph-mon[35525]: pgmap v529: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 430 op/s 2024-09-11T00:27:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:28 smithi183 ceph-mon[35934]: pgmap v530: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-11T00:27:28.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:28 smithi103 ceph-mon[30578]: pgmap v530: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-11T00:27:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:28 smithi117 ceph-mon[35525]: pgmap v530: 129 pgs: 129 active+clean; 58 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-11T00:27:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 -- 172.21.15.103:0/2580694163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c103ad0 msgr2=0x7f889c103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 --2- 172.21.15.103:0/2580694163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c103ad0 0x7f889c103f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8888009890 tx=0x7f888802a370 comp rx=0 tx=0).stop 2024-09-11T00:27:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 -- 172.21.15.103:0/2580694163 shutdown_connections 2024-09-11T00:27:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 --2- 172.21.15.103:0/2580694163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c104490 0x7f889c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 --2- 172.21.15.103:0/2580694163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c103ad0 0x7f889c103f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 --2- 172.21.15.103:0/2580694163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f889c102980 0x7f889c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 -- 172.21.15.103:0/2580694163 >> 172.21.15.103:0/2580694163 conn(0x7f889c0fe240 msgr2=0x7f889c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 -- 172.21.15.103:0/2580694163 shutdown_connections 2024-09-11T00:27:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.057+0000 7f88a3c00640 1 -- 172.21.15.103:0/2580694163 wait complete. 2024-09-11T00:27:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.058+0000 7f88a3c00640 1 Processor -- start 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.058+0000 7f88a3c00640 1 -- start start 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a3c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c102980 0x7f889c19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a3c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f889c103ad0 0x7f889c19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a3c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c104490 0x7f889c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a3c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f889c10e5f0 con 0x7f889c103ad0 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a3c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f889c10e470 con 0x7f889c104490 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a3c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f889c10e770 con 0x7f889c102980 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c102980 0x7f889c19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c102980 0x7f889c19afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39982/0 (socket says 172.21.15.103:39982) 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 -- 172.21.15.103:0/544689119 learned_addr learned my addr 172.21.15.103:0/544689119 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f889be00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f889c103ad0 0x7f889c19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c104490 0x7f889c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 -- 172.21.15.103:0/544689119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c104490 msgr2=0x7f889c19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c104490 0x7f889c19c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 -- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f889c103ad0 msgr2=0x7f889c19b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f889c103ad0 0x7f889c19b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.059+0000 7f88a1200640 1 -- 172.21.15.103:0/544689119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889c19caf0 con 0x7f889c102980 2024-09-11T00:27:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.060+0000 7f88a1200640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c102980 0x7f889c19afb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f889000e9c0 tx=0x7f889000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.060+0000 7f8899600640 1 -- 172.21.15.103:0/544689119 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889000ccc0 con 0x7f889c102980 2024-09-11T00:27:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.060+0000 7f8899600640 1 -- 172.21.15.103:0/544689119 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88900054d0 con 0x7f889c102980 2024-09-11T00:27:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.060+0000 7f8899600640 1 -- 172.21.15.103:0/544689119 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8890010430 con 0x7f889c102980 2024-09-11T00:27:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.060+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889c19cde0 con 0x7f889c102980 2024-09-11T00:27:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.060+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f889c1a34b0 con 0x7f889c102980 2024-09-11T00:27:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 Processor -- start 2024-09-11T00:27:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 -- start start 2024-09-11T00:27:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d56d790 0x55606d56fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 0x55606d5724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d574fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55606d4e06b0 con 0x55606d5700c0 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55606d4cfae0 con 0x55606d56d790 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.064+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55606d4d38f0 con 0x55606d572b00 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d56d790 0x55606d56fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d574fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d574fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39984/0 (socket says 172.21.15.103:39984) 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d56d790 0x55606d56fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40062/0 (socket says 172.21.15.103:40062) 2024-09-11T00:27:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 -- 172.21.15.103:0/2348166946 learned_addr learned my addr 172.21.15.103:0/2348166946 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 -- 172.21.15.103:0/2348166946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d56d790 msgr2=0x55606d56fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 --2- 172.21.15.103:0/2348166946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d56d790 0x55606d56fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 -- 172.21.15.103:0/2348166946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 msgr2=0x55606d5724b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:27:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 --2- 172.21.15.103:0/2348166946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 0x55606d5724b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 -- 172.21.15.103:0/2348166946 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606d49c780 con 0x55606d572b00 2024-09-11T00:27:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f889be00640 1 --2- 172.21.15.103:0/2348166946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d574fc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f888802a890 tx=0x7f8888005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d032220bd5ed6184 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.065+0000 7f8899600640 1 -- 172.21.15.103:0/544689119 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8890004f80 con 0x7f889c102980 2024-09-11T00:27:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.066+0000 7f888e000640 1 -- 172.21.15.103:0/2348166946 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888043020 con 0x55606d572b00 2024-09-11T00:27:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.066+0000 7f888e000640 1 -- 172.21.15.103:0/2348166946 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8888038040 con 0x55606d572b00 2024-09-11T00:27:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.066+0000 7f888e000640 1 -- 172.21.15.103:0/2348166946 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888037040 con 0x55606d572b00 2024-09-11T00:27:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.066+0000 7f88a6ad1740 1 -- 172.21.15.103:0/2348166946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 msgr2=0x55606d574fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.066+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/2348166946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d574fc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f888802a890 tx=0x7f8888005b20 comp rx=0 tx=0).stop 2024-09-11T00:27:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 -- 172.21.15.103:0/2348166946 shutdown_connections 2024-09-11T00:27:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/2348166946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d574fc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/2348166946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 0x55606d5724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/2348166946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d56d790 0x55606d56fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 -- 172.21.15.103:0/2348166946 >> 172.21.15.103:0/2348166946 conn(0x55606d5688e0 msgr2=0x55606d569670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 -- 172.21.15.103:0/2348166946 shutdown_connections 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 -- 172.21.15.103:0/2348166946 wait complete. 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.067+0000 7f88a6ad1740 1 -- start start 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f8899600640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88840b5340 0x7f88840b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 1 Processor -- start 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f8899600640 1 -- 172.21.15.103:0/544689119 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f88900e62b0 con 0x7f889c102980 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a1c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88840b5340 0x7f88840b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d576ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d577560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d578130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55606d407ab0 con 0x55606d56d790 2024-09-11T00:27:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55606d476570 con 0x55606d5700c0 2024-09-11T00:27:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55606d575a80 con 0x55606d572b00 2024-09-11T00:27:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.068+0000 7f88a6ad1740 10 client.0 _ll_register_callbacks cb 0x55606d485100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f88a1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d576ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f889be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d577560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f88a1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d576ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53042/0 (socket says 172.21.15.103:53042) 2024-09-11T00:27:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f88a1200640 1 -- 172.21.15.103:0/3196311766 learned_addr learned my addr 172.21.15.103:0/3196311766 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f88a1c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88840b5340 0x7f88840b7800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f889c19bd30 tx=0x7f8894009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f88a1c00640 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d578130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f889be00640 1 -- 172.21.15.103:0/3196311766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 msgr2=0x55606d578130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f889be00640 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d578130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f889be00640 1 -- 172.21.15.103:0/3196311766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 msgr2=0x55606d576ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f889be00640 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d576ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f889be00640 1 -- 172.21.15.103:0/3196311766 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606d231790 con 0x55606d5700c0 2024-09-11T00:27:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f889be00640 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d577560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8888004650 tx=0x7f8888004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2934eeee004cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f8882a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:27:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.069+0000 7f8882a00640 1 -- 172.21.15.103:0/3196311766 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888804e020 con 0x55606d5700c0 2024-09-11T00:27:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.070+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8888038040 con 0x55606d5700c0 2024-09-11T00:27:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.070+0000 7f8882a00640 1 -- 172.21.15.103:0/3196311766 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8888030730 con 0x55606d5700c0 2024-09-11T00:27:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.070+0000 7f8882a00640 1 -- 172.21.15.103:0/3196311766 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8888059050 con 0x55606d5700c0 2024-09-11T00:27:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.070+0000 7f88a6ad1740 10 client.25808 fetch_fsmap learned FSMap version 35 2024-09-11T00:27:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.070+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55606d2aa620 con 0x55606d5700c0 2024-09-11T00:27:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.070+0000 7f8882a00640 1 -- 172.21.15.103:0/3196311766 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8888038040 con 0x55606d5700c0 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f8882a00640 1 -- 172.21.15.103:0/3196311766 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f88880049c0 con 0x55606d5700c0 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f88a6ad1740 10 client.25808 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f88a6ad1740 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f88a6ad1740 20 client.25808 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d582340 0x55606d584730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f88a6ad1740 4 client.25808 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55606d4e2600 con 0x55606d582340 2024-09-11T00:27:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.071+0000 7f88a1200640 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d582340 0x55606d584730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.072+0000 7f88a1200640 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d582340 0x55606d584730 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba5bc750d5b267d server_cookie=b33085d3a084aea1 in_seq=0 out_seq=0 2024-09-11T00:27:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.072+0000 7f8882a00640 10 client.25808 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.072+0000 7f8882a00640 1 -- 172.21.15.103:0/3196311766 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55606d4e2600 con 0x55606d582340 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.072+0000 7f8882a00640 10 client.25808 handle_command_reply: tid=0 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 client.25808 shutdown 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 msgr2=0x55606d577560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d577560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8888004650 tx=0x7f8888004270 comp rx=0 tx=0).stop 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 shutdown_connections 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d582340 0x55606d584730 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d578130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d577560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3196311766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d576ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 >> 172.21.15.103:0/3196311766 conn(0x55606d5688e0 msgr2=0x55606d567a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f8882a00640 0 client.25808 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 shutdown_connections 2024-09-11T00:27:29.083 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3196311766 wait complete. 2024-09-11T00:27:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.073+0000 7f88a6ad1740 20 client.25808 trim_cache size 0 max 16384 2024-09-11T00:27:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.089+0000 7f88a6ad1740 1 Processor -- start 2024-09-11T00:27:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.089+0000 7f88a6ad1740 1 -- start start 2024-09-11T00:27:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.089+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d569140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.089+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d569680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d576ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1b01c0 con 0x55606d56d790 2024-09-11T00:27:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1ad9d0 con 0x55606d5700c0 2024-09-11T00:27:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1ab820 con 0x55606d572b00 2024-09-11T00:27:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d569140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.094 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d576ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d576ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:39996/0 (socket says 172.21.15.103:39996) 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 -- 172.21.15.103:0/3114121494 learned_addr learned my addr 172.21.15.103:0/3114121494 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f889be00640 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d569680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 -- 172.21.15.103:0/3114121494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 msgr2=0x55606d569680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d569680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 -- 172.21.15.103:0/3114121494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 msgr2=0x55606d569140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d569140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1c00640 1 -- 172.21.15.103:0/3114121494 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606d575e50 con 0x55606d572b00 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.090+0000 7f88a1200640 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d569140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.091+0000 7f88a1c00640 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d576ec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8894003d50 tx=0x7f8894016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34724e51271ecf82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.091+0000 7f888d600640 1 -- 172.21.15.103:0/3114121494 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894003e40 con 0x55606d572b00 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.091+0000 7f888d600640 1 -- 172.21.15.103:0/3114121494 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889401d910 con 0x55606d572b00 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.091+0000 7f888d600640 1 -- 172.21.15.103:0/3114121494 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889401e7c0 con 0x55606d572b00 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3114121494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 msgr2=0x55606d576ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d576ec0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8894003d50 tx=0x7f8894016000 comp rx=0 tx=0).stop 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3114121494 shutdown_connections 2024-09-11T00:27:29.096 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d572b00 0x55606d576ec0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d5700c0 0x55606d569680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/3114121494 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d56d790 0x55606d569140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3114121494 >> 172.21.15.103:0/3114121494 conn(0x55606d5688e0 msgr2=0x55606d572ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3114121494 shutdown_connections 2024-09-11T00:27:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 -- 172.21.15.103:0/3114121494 wait complete. 2024-09-11T00:27:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.092+0000 7f88a6ad1740 1 -- start start 2024-09-11T00:27:29.097 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 1 Processor -- start 2024-09-11T00:27:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d56d790 0x55606d57dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 0x55606d57e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.099 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d572b00 0x55606d57c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1abe90 con 0x55606d5700c0 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1ad300 con 0x55606d572b00 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f889c1ac560 con 0x55606d56d790 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a6ad1740 10 client.0 _ll_register_callbacks cb 0x7f8894009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d56d790 0x55606d57dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d56d790 0x55606d57dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:40012/0 (socket says 172.21.15.103:40012) 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a1200640 1 -- 172.21.15.103:0/1858638044 learned_addr learned my addr 172.21.15.103:0/1858638044 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f88a1c00640 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d572b00 0x55606d57c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.093+0000 7f889be00640 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 0x55606d57e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a1200640 1 -- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d572b00 msgr2=0x55606d57c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a1200640 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d572b00 0x55606d57c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a1200640 1 -- 172.21.15.103:0/1858638044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 msgr2=0x55606d57e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a1200640 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 0x55606d57e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a1200640 1 -- 172.21.15.103:0/1858638044 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606d57ea00 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a1200640 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d56d790 0x55606d57dc30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8890004a60 tx=0x7f889000efb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c645db6d103c5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f8882a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f8882a00640 1 -- 172.21.15.103:0/1858638044 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88900d3070 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f8882a00640 1 -- 172.21.15.103:0/1858638044 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f88900a88f0 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f889401d910 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f8882a00640 1 -- 172.21.15.103:0/1858638044 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f88900b5e60 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a6ad1740 10 client.25852 fetch_fsmap learned FSMap version 35 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.094+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55606d5ef8a0 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f8882a00640 1 -- 172.21.15.103:0/1858638044 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f889401d910 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f8882a00640 1 -- 172.21.15.103:0/1858638044 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 35) ==== 8378+0+0 (secure 0 0 0) 0x7f88900c7150 con 0x55606d56d790 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f88a6ad1740 10 client.25852 fetch_fsmap finished waiting for FSMap version 35 2024-09-11T00:27:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f88a6ad1740 10 client.25852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f88a6ad1740 20 client.25852 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d5f0210 0x55606d5f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f88a6ad1740 4 client.25852 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55606d4de0b0 con 0x55606d5f0210 2024-09-11T00:27:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.095+0000 7f889be00640 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d5f0210 0x55606d5f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.096+0000 7f889be00640 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d5f0210 0x55606d5f2630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0137e435f6949b server_cookie=828043625b90f1bb in_seq=0 out_seq=0 2024-09-11T00:27:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.096+0000 7f8882a00640 10 client.25852 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f8882a00640 1 -- 172.21.15.103:0/1858638044 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55606d4de0b0 con 0x55606d5f0210 2024-09-11T00:27:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f8882a00640 10 client.25852 handle_command_reply: tid=0 2024-09-11T00:27:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 client.25852 shutdown 2024-09-11T00:27:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d56d790 msgr2=0x55606d57dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d56d790 0x55606d57dc30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8890004a60 tx=0x7f889000efb0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 shutdown_connections 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55606d572b00 0x55606d57c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55606d5700c0 0x55606d57e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55606d5f0210 0x55606d5f2630 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 --2- 172.21.15.103:0/1858638044 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55606d56d790 0x55606d57dc30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 >> 172.21.15.103:0/1858638044 conn(0x55606d5688e0 msgr2=0x55606d569140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.106+0000 7f8882a00640 0 client.25852 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.107+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 shutdown_connections 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.107+0000 7f88a6ad1740 1 -- 172.21.15.103:0/1858638044 wait complete. 2024-09-11T00:27:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.107+0000 7f88a6ad1740 20 client.25852 trim_cache size 0 max 16384 2024-09-11T00:27:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:27:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:27:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:27:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88840b5340 msgr2=0x7f88840b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88840b5340 0x7f88840b7800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f889c19bd30 tx=0x7f8894009500 comp rx=0 tx=0).stop 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c102980 msgr2=0x7f889c19afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c102980 0x7f889c19afb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f889000e9c0 tx=0x7f889000ee80 comp rx=0 tx=0).stop 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 shutdown_connections 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f88840b5340 0x7f88840b7800 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f889c104490 0x7f889c19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f889c103ad0 0x7f889c19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.112 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 --2- 172.21.15.103:0/544689119 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f889c102980 0x7f889c19afb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.108+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 >> 172.21.15.103:0/544689119 conn(0x7f889c0fe240 msgr2=0x7f889c075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.109+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 shutdown_connections 2024-09-11T00:27:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:29.109+0000 7f88a3c00640 1 -- 172.21.15.103:0/544689119 wait complete. 2024-09-11T00:27:29.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:29 smithi183 ceph-mon[35934]: daemon mds.f restarted 2024-09-11T00:27:29.781 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:27:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setuser ceph since I am not root 2024-09-11T00:27:29.781 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:27:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setgroup ceph since I am not root 2024-09-11T00:27:29.781 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:27:29 smithi183 ceph-mds[65455]: -- 172.21.15.183:0/2028072840 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641d4901860 con 0x5641d48de800 2024-09-11T00:27:29.781 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:27:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: starting mds.f at 2024-09-11T00:27:29.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:29 smithi103 ceph-mon[30578]: daemon mds.f restarted 2024-09-11T00:27:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:29 smithi117 ceph-mon[35525]: daemon mds.f restarted 2024-09-11T00:27:30.665 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T00:27:30.665 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T00:27:30.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:27:30.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:30 smithi103 ceph-mon[30578]: pgmap v531: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 343 op/s 2024-09-11T00:27:30.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:30 smithi103 ceph-mon[30578]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T00:27:30.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:30 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6834/2940154743,v1:172.21.15.183:6835/2940154743] up:boot 2024-09-11T00:27:30.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:30 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T00:27:30.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:30 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:27:30.722 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:30 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:27:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:30 smithi183 ceph-mon[35934]: pgmap v531: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 343 op/s 2024-09-11T00:27:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:30 smithi183 ceph-mon[35934]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T00:27:30.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:30 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6834/2940154743,v1:172.21.15.183:6835/2940154743] up:boot 2024-09-11T00:27:30.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:30 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T00:27:30.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:30 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:27:30.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:30 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:27:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:30 smithi117 ceph-mon[35525]: pgmap v531: 129 pgs: 129 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 343 op/s 2024-09-11T00:27:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:30 smithi117 ceph-mon[35525]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T00:27:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:30 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6834/2940154743,v1:172.21.15.183:6835/2940154743] up:boot 2024-09-11T00:27:30.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:30 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-09-11T00:27:30.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:30 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:27:30.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:30 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:27:31.105 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:27:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:32 smithi183 ceph-mon[35934]: pgmap v533: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:32.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:32 smithi103 ceph-mon[30578]: pgmap v533: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:32.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:32 smithi117 ceph-mon[35525]: pgmap v533: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 -- 172.21.15.103:0/4164962101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e20104e10 msgr2=0x7f0e201051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 --2- 172.21.15.103:0/4164962101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e20104e10 0x7f0e201051f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009900 tx=0x7f0e1402a400 comp rx=0 tx=0).stop 2024-09-11T00:27:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 -- 172.21.15.103:0/4164962101 shutdown_connections 2024-09-11T00:27:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 --2- 172.21.15.103:0/4164962101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e2010b1a0 0x7f0e201119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 --2- 172.21.15.103:0/4164962101 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e2010a820 0x7f0e2010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 --2- 172.21.15.103:0/4164962101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e20104e10 0x7f0e201051f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 -- 172.21.15.103:0/4164962101 >> 172.21.15.103:0/4164962101 conn(0x7f0e20100840 msgr2=0x7f0e20102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 -- 172.21.15.103:0/4164962101 shutdown_connections 2024-09-11T00:27:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.557+0000 7f0e26400640 1 -- 172.21.15.103:0/4164962101 wait complete. 2024-09-11T00:27:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.558+0000 7f0e26400640 1 Processor -- start 2024-09-11T00:27:33.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.559+0000 7f0e26400640 1 -- start start 2024-09-11T00:27:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.559+0000 7f0e26400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e20104e10 0x7f0e2019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.559+0000 7f0e26400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2010a820 0x7f0e2019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.559+0000 7f0e26400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e2010b1a0 0x7f0e201a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.559+0000 7f0e26400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20114eb0 con 0x7f0e2010a820 2024-09-11T00:27:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.559+0000 7f0e26400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20114d30 con 0x7f0e20104e10 2024-09-11T00:27:33.563 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.559+0000 7f0e26400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0e20115030 con 0x7f0e2010b1a0 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e20104e10 0x7f0e2019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e20104e10 0x7f0e2019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42582/0 (socket says 172.21.15.103:42582) 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 -- 172.21.15.103:0/3110181140 learned_addr learned my addr 172.21.15.103:0/3110181140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25a00640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e2010b1a0 0x7f0e201a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 -- 172.21.15.103:0/3110181140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e2010b1a0 msgr2=0x7f0e201a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e2010b1a0 0x7f0e201a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 -- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2010a820 msgr2=0x7f0e2019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2010a820 0x7f0e2019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 -- 172.21.15.103:0/3110181140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e201a1300 con 0x7f0e20104e10 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.560+0000 7f0e25000640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e20104e10 0x7f0e2019b250 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005ab0 tx=0x7f0e14003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.561+0000 7f0e1d600640 1 -- 172.21.15.103:0/3110181140 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1403f070 con 0x7f0e20104e10 2024-09-11T00:27:33.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.561+0000 7f0e1d600640 1 -- 172.21.15.103:0/3110181140 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e14031b60 con 0x7f0e20104e10 2024-09-11T00:27:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.561+0000 7f0e1d600640 1 -- 172.21.15.103:0/3110181140 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14035030 con 0x7f0e20104e10 2024-09-11T00:27:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.561+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e201a1590 con 0x7f0e20104e10 2024-09-11T00:27:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.561+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e201a7cb0 con 0x7f0e20104e10 2024-09-11T00:27:33.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.562+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e201051f0 con 0x7f0e20104e10 2024-09-11T00:27:33.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.565+0000 7f0e1d600640 1 -- 172.21.15.103:0/3110181140 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0e1404c060 con 0x7f0e20104e10 2024-09-11T00:27:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.566+0000 7f0e1d600640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e080b5390 0x7f0e080b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.567+0000 7f0e1d600640 1 -- 172.21.15.103:0/3110181140 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f0e141068e0 con 0x7f0e20104e10 2024-09-11T00:27:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.567+0000 7f0e1fe00640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e080b5390 0x7f0e080b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:33.569 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.568+0000 7f0e1fe00640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e080b5390 0x7f0e080b7850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005d00 tx=0x7f0e0c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:33.570 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.568+0000 7f0e1d600640 1 -- 172.21.15.103:0/3110181140 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0e140cd3b0 con 0x7f0e20104e10 2024-09-11T00:27:33.850 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.848+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e201a7f90 con 0x7f0e20104e10 2024-09-11T00:27:33.851 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.849+0000 7f0e1d600640 1 -- 172.21.15.103:0/3110181140 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11942 (secure 0 0 0) 0x7f0e140cccb0 con 0x7f0e20104e10 2024-09-11T00:27:33.851 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:27:33.852 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":37,"btime":"2024-09-11T00:27:29:450603+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:27:29.450591+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25858":{"gid":25858,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2940154743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2940154743},{"type":"v1","addr":"172.21.15.183:6835","nonce":2940154743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:27:33.852 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 37 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e080b5390 msgr2=0x7f0e080b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e080b5390 0x7f0e080b7850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c005d00 tx=0x7f0e0c005c50 comp rx=0 tx=0).stop 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e20104e10 msgr2=0x7f0e2019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e20104e10 0x7f0e2019b250 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e14005ab0 tx=0x7f0e14003c10 comp rx=0 tx=0).stop 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 shutdown_connections 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f0e080b5390 0x7f0e080b7850 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0e2010b1a0 0x7f0e201a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e2010a820 0x7f0e2019b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 --2- 172.21.15.103:0/3110181140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e20104e10 0x7f0e2019b250 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.852+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 >> 172.21.15.103:0/3110181140 conn(0x7f0e20100840 msgr2=0x7f0e20102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:33.855 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.853+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 shutdown_connections 2024-09-11T00:27:33.856 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:33.853+0000 7f0e26400640 1 -- 172.21.15.103:0/3110181140 wait complete. 2024-09-11T00:27:34.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:34 smithi103 ceph-mon[30578]: pgmap v534: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 347 op/s 2024-09-11T00:27:34.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:34 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3110181140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:27:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:34 smithi183 ceph-mon[35934]: pgmap v534: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 347 op/s 2024-09-11T00:27:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:34 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3110181140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:27:34.833 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T00:27:34.834 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T00:27:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:34 smithi117 ceph-mon[35525]: pgmap v534: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 347 op/s 2024-09-11T00:27:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:34 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3110181140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:27:35.038 INFO:tasks.workunit.client.0.smithi103.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-09-11T00:27:35.039 INFO:tasks.workunit.client.0.smithi103.stdout:All 50000 operations completed A-OK! 2024-09-11T00:27:35.039 INFO:tasks.workunit.client.0.smithi103.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-09-11T00:27:35.040 INFO:tasks.workunit.client.0.smithi103.stdout:Seed set to 1 2024-09-11T00:27:35.182 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-11T00:27:35.182 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-11T00:27:35.184 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-11T00:27:35.184 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-11T00:27:35.184 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support clone range, disabling! 2024-09-11T00:27:35.186 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-11T00:27:35.188 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support exchange range, disabling! 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero length punch hole 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero length punch hole 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero length punch hole 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero size read 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero length punch hole 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero size read 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero length punch hole 2024-09-11T00:27:35.189 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x4310e9 2024-09-11T00:27:35.196 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0x954a45 2024-09-11T00:27:35.255 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:27:35.273 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x9bc582 2024-09-11T00:27:35.318 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x9eecbc 2024-09-11T00:27:35.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 -- 172.21.15.103:0/3229605721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 msgr2=0x7f7984075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 --2- 172.21.15.103:0/3229605721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 0x7f7984075920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7974009ca0 tx=0x7f797402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 -- 172.21.15.103:0/3229605721 shutdown_connections 2024-09-11T00:27:35.930 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 --2- 172.21.15.103:0/3229605721 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f798410f3c0 0x7f79841117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 --2- 172.21.15.103:0/3229605721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7984075f40 0x7f798410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 --2- 172.21.15.103:0/3229605721 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 0x7f7984075920 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 -- 172.21.15.103:0/3229605721 >> 172.21.15.103:0/3229605721 conn(0x7f79840fe810 msgr2=0x7f7984100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 -- 172.21.15.103:0/3229605721 shutdown_connections 2024-09-11T00:27:35.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.928+0000 7f7989a00640 1 -- 172.21.15.103:0/3229605721 wait complete. 2024-09-11T00:27:35.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.929+0000 7f7989a00640 1 Processor -- start 2024-09-11T00:27:35.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.929+0000 7f7989a00640 1 -- start start 2024-09-11T00:27:35.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7989a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 0x7f798419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7989a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7984075f40 0x7f798419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7989a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f798410f3c0 0x7f79841a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7989a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7984114f50 con 0x7f7984075f40 2024-09-11T00:27:35.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7989a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7984114dd0 con 0x7f7984075540 2024-09-11T00:27:35.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7989a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79841150d0 con 0x7f798410f3c0 2024-09-11T00:27:35.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7983400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f798410f3c0 0x7f79841a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7982a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 0x7f798419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7983400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f798410f3c0 0x7f79841a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:56964/0 (socket says 172.21.15.103:56964) 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7983400640 1 -- 172.21.15.103:0/3393098419 learned_addr learned my addr 172.21.15.103:0/3393098419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.930+0000 7f7982000640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7984075f40 0x7f798419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7982a00640 1 -- 172.21.15.103:0/3393098419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f798410f3c0 msgr2=0x7f79841a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7982a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f798410f3c0 0x7f79841a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7982a00640 1 -- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7984075f40 msgr2=0x7f798419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7982a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7984075f40 0x7f798419b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7982a00640 1 -- 172.21.15.103:0/3393098419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79841a1510 con 0x7f7984075540 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7983400640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f798410f3c0 0x7f79841a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7982a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 0x7f798419b460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f797402ade0 tx=0x7f7974031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f797b400640 1 -- 172.21.15.103:0/3393098419 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7974003b70 con 0x7f7984075540 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f797b400640 1 -- 172.21.15.103:0/3393098419 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7974031910 con 0x7f7984075540 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.931+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79841a17a0 con 0x7f7984075540 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.932+0000 7f797b400640 1 -- 172.21.15.103:0/3393098419 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797402f950 con 0x7f7984075540 2024-09-11T00:27:35.934 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.932+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7984102980 con 0x7f7984075540 2024-09-11T00:27:35.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.936+0000 7f798bffd740 1 Processor -- start 2024-09-11T00:27:35.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f798bffd740 1 -- start start 2024-09-11T00:27:35.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f798bffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f798bffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2aa0 0x55c2db2ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f798bffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2f0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f798bffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2db257ca0 con 0x55c2db2e2aa0 2024-09-11T00:27:35.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f798bffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2db2470d0 con 0x55c2db2e2180 2024-09-11T00:27:35.939 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f798bffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2db24aee0 con 0x55c2db2eea30 2024-09-11T00:27:35.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f7983400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.937+0000 7f797b400640 1 -- 172.21.15.103:0/3393098419 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f797402faf0 con 0x7f7984075540 2024-09-11T00:27:35.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2e2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42604/0 (socket says 172.21.15.103:42604) 2024-09-11T00:27:35.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 -- 172.21.15.103:0/117742925 learned_addr learned my addr 172.21.15.103:0/117742925 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:35.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 -- 172.21.15.103:0/117742925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 msgr2=0x55c2db2f0e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:35.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 --2- 172.21.15.103:0/117742925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2f0e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 -- 172.21.15.103:0/117742925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2aa0 msgr2=0x55c2db2ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 --2- 172.21.15.103:0/117742925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2aa0 0x55c2db2ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 -- 172.21.15.103:0/117742925 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2db1f9400 con 0x55c2db2e2180 2024-09-11T00:27:35.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.938+0000 7f7983400640 1 --2- 172.21.15.103:0/117742925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2e2560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f797c011570 tx=0x7f797c011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3547b1a0169ef233 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.939+0000 7f7973e00640 1 -- 172.21.15.103:0/117742925 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c01e070 con 0x55c2db2e2180 2024-09-11T00:27:35.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.939+0000 7f7973e00640 1 -- 172.21.15.103:0/117742925 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797c00faf0 con 0x55c2db2e2180 2024-09-11T00:27:35.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.939+0000 7f7973e00640 1 -- 172.21.15.103:0/117742925 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c019bd0 con 0x55c2db2e2180 2024-09-11T00:27:35.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f798bffd740 1 -- 172.21.15.103:0/117742925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 msgr2=0x55c2db2e2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f798bffd740 1 --2- 172.21.15.103:0/117742925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2e2560 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f797c011570 tx=0x7f797c011a30 comp rx=0 tx=0).stop 2024-09-11T00:27:35.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f797b400640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79680b5360 0x7f79680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f797b400640 1 -- 172.21.15.103:0/3393098419 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f7974106f40 con 0x7f7984075540 2024-09-11T00:27:35.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f798bffd740 1 -- 172.21.15.103:0/117742925 shutdown_connections 2024-09-11T00:27:35.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f798bffd740 1 --2- 172.21.15.103:0/117742925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2f0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f798bffd740 1 --2- 172.21.15.103:0/117742925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2aa0 0x55c2db2ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f798bffd740 1 --2- 172.21.15.103:0/117742925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2e2560 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.940+0000 7f798bffd740 1 -- 172.21.15.103:0/117742925 >> 172.21.15.103:0/117742925 conn(0x55c2db2dffb0 msgr2=0x55c2db2e1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.941+0000 7f798bffd740 1 -- 172.21.15.103:0/117742925 shutdown_connections 2024-09-11T00:27:35.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.941+0000 7f7983400640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79680b5360 0x7f79680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.942+0000 7f798bffd740 1 -- 172.21.15.103:0/117742925 wait complete. 2024-09-11T00:27:35.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.942+0000 7f7983400640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79680b5360 0x7f79680b7820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f797c0196b0 tx=0x7f797c019710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.944 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.942+0000 7f798bffd740 1 -- start start 2024-09-11T00:27:35.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 1 Processor -- start 2024-09-11T00:27:35.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2eb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.945 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c2db0a87a0 con 0x55c2db2e2180 2024-09-11T00:27:35.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c2db1eca10 con 0x55c2db2e2aa0 2024-09-11T00:27:35.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2db2f1700 con 0x55c2db2eea30 2024-09-11T00:27:35.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.943+0000 7f798bffd740 10 client.0 _ll_register_callbacks cb 0x55c2db1fcbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:35.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2eb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7983400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2eb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42616/0 (socket says 172.21.15.103:42616) 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 -- 172.21.15.103:0/530812221 learned_addr learned my addr 172.21.15.103:0/530812221 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982a00640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 -- 172.21.15.103:0/530812221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 msgr2=0x55c2db2ebbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2ebbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 -- 172.21.15.103:0/530812221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 msgr2=0x55c2db2ee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2ee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982000640 1 -- 172.21.15.103:0/530812221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2dafb4b90 con 0x55c2db2e2aa0 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7983400640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:35.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.944+0000 7f7982a00640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:35.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f7982000640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2eb670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f796c009be0 tx=0x7f796c00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c7f1441a454a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.948 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f7970c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f7970c00640 1 -- 172.21.15.103:0/530812221 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c022030 con 0x55c2db2e2aa0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f797c00faf0 con 0x55c2db2e2aa0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f7970c00640 1 -- 172.21.15.103:0/530812221 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f796c004020 con 0x55c2db2e2aa0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f7970c00640 1 -- 172.21.15.103:0/530812221 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f796c02c050 con 0x55c2db2e2aa0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f798bffd740 10 client.25838 fetch_fsmap learned FSMap version 37 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.945+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c2db2fd0a0 con 0x55c2db2e2aa0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f7970c00640 1 -- 172.21.15.103:0/530812221 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797c00faf0 con 0x55c2db2e2aa0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f7970c00640 1 -- 172.21.15.103:0/530812221 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7f796c027400 con 0x55c2db2e2aa0 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f798bffd740 10 client.25838 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f798bffd740 10 client.25838 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f798bffd740 20 client.25838 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f798bffd740 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db2fe500 0x55c2db300930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f798bffd740 4 client.25838 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:35.949 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2db259bf0 con 0x55c2db2fe500 2024-09-11T00:27:35.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.946+0000 7f7982a00640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db2fe500 0x55c2db300930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.950 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.947+0000 7f7982a00640 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db2fe500 0x55c2db300930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca922b275a61035 server_cookie=ec2b4928eaf6e53d in_seq=0 out_seq=0 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.947+0000 7f7970c00640 10 client.25838 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.948+0000 7f7970c00640 1 -- 172.21.15.103:0/530812221 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2db259bf0 con 0x55c2db2fe500 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.948+0000 7f7970c00640 10 client.25838 handle_command_reply: tid=0 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.948+0000 7f798bffd740 1 client.25838 shutdown 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 msgr2=0x55c2db2eb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2eb670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f796c009be0 tx=0x7f796c00b690 comp rx=0 tx=0).stop 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 shutdown_connections 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db2fe500 0x55c2db300930 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2ebbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2eb670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 --2- 172.21.15.103:0/530812221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2ee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.951 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 >> 172.21.15.103:0/530812221 conn(0x55c2db2dffb0 msgr2=0x55c2db2e5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f7970c00640 0 client.25838 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:35.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 shutdown_connections 2024-09-11T00:27:35.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 1 -- 172.21.15.103:0/530812221 wait complete. 2024-09-11T00:27:35.952 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.949+0000 7f798bffd740 20 client.25838 trim_cache size 0 max 16384 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 Processor -- start 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 -- start start 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2e0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79841b0240 con 0x55c2db2e2180 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ada50 con 0x55c2db2e2aa0 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f798bffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ab900 con 0x55c2db2eea30 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f7982000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f7982a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2e0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f7982000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2df8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42626/0 (socket says 172.21.15.103:42626) 2024-09-11T00:27:35.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.974+0000 7f7982000640 1 -- 172.21.15.103:0/794964976 learned_addr learned my addr 172.21.15.103:0/794964976 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:35.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7983400640 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7982000640 1 -- 172.21.15.103:0/794964976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 msgr2=0x55c2db2e9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7982000640 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2e9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7982000640 1 -- 172.21.15.103:0/794964976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 msgr2=0x55c2db2e0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7982000640 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2e0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7982000640 1 -- 172.21.15.103:0/794964976 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2db2e1690 con 0x55c2db2e2aa0 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7982000640 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2df8a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f796c0277f0 tx=0x7f796c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445e4c7184617a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7973400640 1 -- 172.21.15.103:0/794964976 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c022040 con 0x55c2db2e2aa0 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7973400640 1 -- 172.21.15.103:0/794964976 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f796c027400 con 0x55c2db2e2aa0 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.975+0000 7f7973400640 1 -- 172.21.15.103:0/794964976 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c0035f0 con 0x55c2db2e2aa0 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.976+0000 7f798bffd740 1 -- 172.21.15.103:0/794964976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 msgr2=0x55c2db2df8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.976+0000 7f798bffd740 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2df8a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f796c0277f0 tx=0x7f796c009fd0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.976+0000 7f798bffd740 1 -- 172.21.15.103:0/794964976 shutdown_connections 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.976+0000 7f798bffd740 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2eea30 0x55c2db2e9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.976+0000 7f798bffd740 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2aa0 0x55c2db2df8a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.976+0000 7f798bffd740 1 --2- 172.21.15.103:0/794964976 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2e2180 0x55c2db2e0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.976+0000 7f798bffd740 1 -- 172.21.15.103:0/794964976 >> 172.21.15.103:0/794964976 conn(0x55c2db2dffb0 msgr2=0x55c2db2e6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:35.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.977+0000 7f798bffd740 1 -- 172.21.15.103:0/794964976 shutdown_connections 2024-09-11T00:27:35.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.977+0000 7f798bffd740 1 -- 172.21.15.103:0/794964976 wait complete. 2024-09-11T00:27:35.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.978+0000 7f798bffd740 1 -- start start 2024-09-11T00:27:35.980 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.978+0000 7f798bffd740 1 Processor -- start 2024-09-11T00:27:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f798bffd740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f798bffd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2e2aa0 0x55c2db2f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f7982a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f798bffd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2eea30 0x55c2db2f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f798bffd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79841abf50 con 0x55c2db2eea30 2024-09-11T00:27:35.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f798bffd740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ad380 con 0x55c2db2e2180 2024-09-11T00:27:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f7982a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42634/0 (socket says 172.21.15.103:42634) 2024-09-11T00:27:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f798bffd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79841ac5e0 con 0x55c2db2e2aa0 2024-09-11T00:27:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f7982a00640 1 -- 172.21.15.103:0/2246740883 learned_addr learned my addr 172.21.15.103:0/2246740883 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f798bffd740 10 client.0 _ll_register_callbacks cb 0x7f796c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:35.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.979+0000 7f7982000640 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2e2aa0 0x55c2db2f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7983400640 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2eea30 0x55c2db2f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7982a00640 1 -- 172.21.15.103:0/2246740883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2e2aa0 msgr2=0x55c2db2f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7982a00640 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2e2aa0 0x55c2db2f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7982a00640 1 -- 172.21.15.103:0/2246740883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2eea30 msgr2=0x55c2db2f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7982a00640 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2eea30 0x55c2db2f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7982a00640 1 -- 172.21.15.103:0/2246740883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2db2f35e0 con 0x55c2db2e2180 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7982a00640 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2f72d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7974000f80 tx=0x7f7974004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d914d025e68b0c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7970c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:27:35.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7970c00640 1 -- 172.21.15.103:0/2246740883 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79740d6040 con 0x55c2db2e2180 2024-09-11T00:27:35.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f7970c00640 1 -- 172.21.15.103:0/2246740883 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f7974003d10 con 0x55c2db2e2180 2024-09-11T00:27:35.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.980+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f796c027400 con 0x55c2db2e2180 2024-09-11T00:27:35.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.981+0000 7f7970c00640 1 -- 172.21.15.103:0/2246740883 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f79740cda10 con 0x55c2db2e2180 2024-09-11T00:27:35.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.981+0000 7f798bffd740 10 client.25844 fetch_fsmap learned FSMap version 37 2024-09-11T00:27:35.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.981+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c2db36b5b0 con 0x55c2db2e2180 2024-09-11T00:27:35.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f7970c00640 1 -- 172.21.15.103:0/2246740883 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f796c027400 con 0x55c2db2e2180 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f7970c00640 1 -- 172.21.15.103:0/2246740883 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7f79740c9580 con 0x55c2db2e2180 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f798bffd740 10 client.25844 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f798bffd740 10 client.25844 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f798bffd740 20 client.25844 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f798bffd740 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db36bd30 0x55c2db36e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f798bffd740 4 client.25844 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.982+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55c2db2556a0 con 0x55c2db36bd30 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.983+0000 7f7982000640 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db36bd30 0x55c2db36e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.983+0000 7f7982000640 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db36bd30 0x55c2db36e150 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b985bc5d100ed55f server_cookie=788d22189140287b in_seq=0 out_seq=0 2024-09-11T00:27:35.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:35.983+0000 7f7970c00640 10 client.25844 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:36 smithi183 ceph-mon[35934]: pgmap v535: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f7970c00640 1 -- 172.21.15.103:0/2246740883 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c2db2556a0 con 0x55c2db36bd30 2024-09-11T00:27:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f7970c00640 10 client.25844 handle_command_reply: tid=0 2024-09-11T00:27:36.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 client.25844 shutdown 2024-09-11T00:27:36.891 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:36 smithi103 ceph-mon[30578]: pgmap v535: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 msgr2=0x55c2db2f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2f72d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7974000f80 tx=0x7f7974004a40 comp rx=0 tx=0).stop 2024-09-11T00:27:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 shutdown_connections 2024-09-11T00:27:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c2db2eea30 0x55c2db2f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c2db36bd30 0x55c2db36e150 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2db2e2aa0 0x55c2db2f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 --2- 172.21.15.103:0/2246740883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c2db2e2180 0x55c2db2f72d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 >> 172.21.15.103:0/2246740883 conn(0x55c2db2dffb0 msgr2=0x55c2db2e9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f7970c00640 0 client.25844 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 shutdown_connections 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 1 -- 172.21.15.103:0/2246740883 wait complete. 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.889+0000 7f798bffd740 20 client.25844 trim_cache size 0 max 16384 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:27:36.893 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 37, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 14, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "76cb9b6b": { 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 14, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:27:36.894 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:27:36.895 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79680b5360 msgr2=0x7f79680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79680b5360 0x7f79680b7820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f797c0196b0 tx=0x7f797c019710 comp rx=0 tx=0).stop 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 msgr2=0x7f798419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 0x7f798419b460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f797402ade0 tx=0x7f7974031660 comp rx=0 tx=0).stop 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 shutdown_connections 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79680b5360 0x7f79680b7820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f798410f3c0 0x7f79841a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7984075f40 0x7f798419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 --2- 172.21.15.103:0/3393098419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7984075540 0x7f798419b460 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:36.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 >> 172.21.15.103:0/3393098419 conn(0x7f79840fe810 msgr2=0x7f79841115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 shutdown_connections 2024-09-11T00:27:36.897 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:27:36.890+0000 7f7989a00640 1 -- 172.21.15.103:0/3393098419 wait complete. 2024-09-11T00:27:36.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:36 smithi117 ceph-mon[35525]: pgmap v535: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:37.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:37 smithi103 ceph-mon[30578]: pgmap v536: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:37.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:37 smithi183 ceph-mon[35934]: pgmap v536: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:37.786 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '76cb9b6b', quiesce took 7.1 seconds. Will release after: 22.9 2024-09-11T00:27:37.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:37 smithi117 ceph-mon[35525]: pgmap v536: 129 pgs: 129 active+clean; 57 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 346 op/s 2024-09-11T00:27:39.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:39 smithi183 ceph-mon[35934]: pgmap v537: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 348 op/s 2024-09-11T00:27:39.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:39 smithi103 ceph-mon[30578]: pgmap v537: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 348 op/s 2024-09-11T00:27:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:39 smithi117 ceph-mon[35525]: pgmap v537: 129 pgs: 129 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 348 op/s 2024-09-11T00:27:41.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:41 smithi183 ceph-mon[35934]: pgmap v538: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-09-11T00:27:41.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:41 smithi103 ceph-mon[30578]: pgmap v538: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-09-11T00:27:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:41 smithi117 ceph-mon[35525]: pgmap v538: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-09-11T00:27:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.226+0000 7f090e000640 1 -- 172.21.15.103:0/2169868038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 msgr2=0x7f090810d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.226+0000 7f090e000640 1 --2- 172.21.15.103:0/2169868038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 0x7f090810d030 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0900009cd0 tx=0x7f090002a8e0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.227+0000 7f090e000640 1 -- 172.21.15.103:0/2169868038 shutdown_connections 2024-09-11T00:27:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.227+0000 7f090e000640 1 --2- 172.21.15.103:0/2169868038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 0x7f090810d030 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.227+0000 7f090e000640 1 --2- 172.21.15.103:0/2169868038 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0908105cf0 0x7f0908106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.227+0000 7f090e000640 1 --2- 172.21.15.103:0/2169868038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908104ba0 0x7f0908104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.227+0000 7f090e000640 1 -- 172.21.15.103:0/2169868038 >> 172.21.15.103:0/2169868038 conn(0x7f0908100460 msgr2=0x7f0908102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.227+0000 7f090e000640 1 -- 172.21.15.103:0/2169868038 shutdown_connections 2024-09-11T00:27:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.227+0000 7f090e000640 1 -- 172.21.15.103:0/2169868038 wait complete. 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 Processor -- start 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 -- start start 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0908104ba0 0x7f090819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908105cf0 0x7f090819b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 0x7f090819c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09081107f0 con 0x7f09081066b0 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0908110670 con 0x7f0908105cf0 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.228+0000 7f090e000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0908110970 con 0x7f0908104ba0 2024-09-11T00:27:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908105cf0 0x7f090819b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0907e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 0x7f090819c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0907400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0908104ba0 0x7f090819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908105cf0 0x7f090819b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44144/0 (socket says 172.21.15.103:44144) 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 -- 172.21.15.103:0/3140660398 learned_addr learned my addr 172.21.15.103:0/3140660398 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 -- 172.21.15.103:0/3140660398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0908104ba0 msgr2=0x7f090819af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0908104ba0 0x7f090819af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 -- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 msgr2=0x7f090819c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 0x7f090819c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 -- 172.21.15.103:0/3140660398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090819ca50 con 0x7f0908105cf0 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.229+0000 7f0906a00640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908105cf0 0x7f090819b560 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e9f0 tx=0x7f08f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.230+0000 7f08ffe00640 1 -- 172.21.15.103:0/3140660398 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f400cd80 con 0x7f0908105cf0 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.230+0000 7f08ffe00640 1 -- 172.21.15.103:0/3140660398 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f40108e0 con 0x7f0908105cf0 2024-09-11T00:27:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.230+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090819cd40 con 0x7f0908105cf0 2024-09-11T00:27:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.230+0000 7f08ffe00640 1 -- 172.21.15.103:0/3140660398 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f4018980 con 0x7f0908105cf0 2024-09-11T00:27:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.230+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081a3410 con 0x7f0908105cf0 2024-09-11T00:27:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 Processor -- start 2024-09-11T00:27:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 -- start start 2024-09-11T00:27:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c67115b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c671184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 0x555c6711afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c670866b0 con 0x555c671160c0 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c67075ae0 con 0x555c67113790 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0910e2f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c670798f0 con 0x555c67118b00 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0907e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c67115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0907e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c67115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44148/0 (socket says 172.21.15.103:44148) 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0907e00640 1 -- 172.21.15.103:0/3045388281 learned_addr learned my addr 172.21.15.103:0/3045388281 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.233+0000 7f0907400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c671184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f08ffe00640 1 -- 172.21.15.103:0/3140660398 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f08f4010430 con 0x7f0908105cf0 2024-09-11T00:27:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f0907e00640 1 -- 172.21.15.103:0/3045388281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 msgr2=0x555c6711afc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f0907e00640 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 0x555c6711afc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f0907e00640 1 -- 172.21.15.103:0/3045388281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 msgr2=0x555c671184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f0907e00640 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c671184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f0907e00640 1 -- 172.21.15.103:0/3045388281 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c67042780 con 0x555c67113790 2024-09-11T00:27:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f0907400640 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c671184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:42.238 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f0907e00640 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c67115b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f090002ae00 tx=0x7f0900009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd754b9ac99c01ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.234+0000 7f08f3e00640 1 -- 172.21.15.103:0/3045388281 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900031430 con 0x555c67113790 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f08f3e00640 1 -- 172.21.15.103:0/3045388281 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0900036040 con 0x555c67113790 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f08f3e00640 1 -- 172.21.15.103:0/3045388281 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900004330 con 0x555c67113790 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 -- 172.21.15.103:0/3045388281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 msgr2=0x555c67115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c67115b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f090002ae00 tx=0x7f0900009a20 comp rx=0 tx=0).stop 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 -- 172.21.15.103:0/3045388281 shutdown_connections 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 0x555c6711afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c671184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 --2- 172.21.15.103:0/3045388281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c67115b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 -- 172.21.15.103:0/3045388281 >> 172.21.15.103:0/3045388281 conn(0x555c6710e8e0 msgr2=0x555c6710f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f08ffe00640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f08ec0b52e0 0x7f08ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.235+0000 7f0910e2f740 1 -- 172.21.15.103:0/3045388281 shutdown_connections 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.236+0000 7f0910e2f740 1 -- 172.21.15.103:0/3045388281 wait complete. 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.236+0000 7f08ffe00640 1 -- 172.21.15.103:0/3140660398 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f08f40e6250 con 0x7f0908105cf0 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.236+0000 7f0907e00640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f08ec0b52e0 0x7f08ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.236+0000 7f0910e2f740 1 -- start start 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.236+0000 7f0910e2f740 1 Processor -- start 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.236+0000 7f0907e00640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f08ec0b52e0 0x7f08ec0b77a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0900009960 tx=0x7f090003d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c6711d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 0x555c6711e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0910e2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555c66fadab0 con 0x555c671160c0 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0910e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x555c6701c570 con 0x555c67113790 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0910e2f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555c6711ba20 con 0x555c67118b00 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0910e2f740 10 client.0 _ll_register_callbacks cb 0x555c6702b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0907400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0907400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44152/0 (socket says 172.21.15.103:44152) 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0907400640 1 -- 172.21.15.103:0/1840444154 learned_addr learned my addr 172.21.15.103:0/1840444154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0907400640 1 -- 172.21.15.103:0/1840444154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 msgr2=0x555c6711e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0907400640 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 0x555c6711e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.237+0000 7f0907400640 1 -- 172.21.15.103:0/1840444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 msgr2=0x555c6711d560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f0906a00640 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c6711d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f0907400640 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c6711d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f0907400640 1 -- 172.21.15.103:0/1840444154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c66dd7790 con 0x555c67113790 2024-09-11T00:27:42.244 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f0906a00640 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c6711d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f0907400640 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711cec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f08f8012760 tx=0x7f08f8012c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3029ef6461a05541 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f08f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f08f0c00640 1 -- 172.21.15.103:0/1840444154 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f800f920 con 0x555c67113790 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0900036040 con 0x555c67113790 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.238+0000 7f08f0c00640 1 -- 172.21.15.103:0/1840444154 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f08f800fac0 con 0x555c67113790 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f08f0c00640 1 -- 172.21.15.103:0/1840444154 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f08f800fcc0 con 0x555c67113790 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 10 client.25856 fetch_fsmap learned FSMap version 37 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555c66e50620 con 0x555c67113790 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f08f0c00640 1 -- 172.21.15.103:0/1840444154 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0900036040 con 0x555c67113790 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f08f0c00640 1 -- 172.21.15.103:0/1840444154 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7f08f801c420 con 0x555c67113790 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 10 client.25856 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:42.245 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 20 client.25856 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c6712ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 4 client.25856 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c67088600 con 0x555c67128930 2024-09-11T00:27:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.239+0000 7f0906a00640 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c6712ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.240+0000 7f0906a00640 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c6712ad70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b502bb08b9d67d server_cookie=4e716badb6e43fb6 in_seq=0 out_seq=0 2024-09-11T00:27:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.240+0000 7f08f0c00640 10 client.25856 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.240+0000 7f08f0c00640 1 -- 172.21.15.103:0/1840444154 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c67088600 con 0x555c67128930 2024-09-11T00:27:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.240+0000 7f08f0c00640 10 client.25856 handle_command_reply: tid=0 2024-09-11T00:27:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 client.25856 shutdown 2024-09-11T00:27:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 msgr2=0x555c6711cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711cec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f08f8012760 tx=0x7f08f8012c20 comp rx=0 tx=0).stop 2024-09-11T00:27:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 shutdown_connections 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c6712ad70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67118b00 0x555c6711e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c671160c0 0x555c6711d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f08f0c00640 0 client.25856 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1840444154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711cec0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 >> 172.21.15.103:0/1840444154 conn(0x555c6710e8e0 msgr2=0x555c6710da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 shutdown_connections 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 1 -- 172.21.15.103:0/1840444154 wait complete. 2024-09-11T00:27:42.248 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.241+0000 7f0910e2f740 20 client.25856 trim_cache size 0 max 16384 2024-09-11T00:27:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.259+0000 7f0910e2f740 1 Processor -- start 2024-09-11T00:27:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.259+0000 7f0910e2f740 1 -- start start 2024-09-11T00:27:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c671160c0 0x555c6711c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c67118ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0910e2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09081b0150 con 0x555c67118b00 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0910e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09081ad960 con 0x555c67113790 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0910e2f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09081ab7b0 con 0x555c671160c0 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c67118ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44156/0 (socket says 172.21.15.103:44156) 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 -- 172.21.15.103:0/1173488951 learned_addr learned my addr 172.21.15.103:0/1173488951 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0906a00640 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c671160c0 0x555c6711c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 -- 172.21.15.103:0/1173488951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c671160c0 msgr2=0x555c6711c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c671160c0 0x555c6711c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 -- 172.21.15.103:0/1173488951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 msgr2=0x555c67118ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c67118ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.264 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.260+0000 7f0907400640 1 -- 172.21.15.103:0/1173488951 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c67112990 con 0x555c67113790 2024-09-11T00:27:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.261+0000 7f0907400640 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711c060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f08f801c910 tx=0x7f08f800f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578c36f18ea6d6b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.261+0000 7f08f3400640 1 -- 172.21.15.103:0/1173488951 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f800f920 con 0x555c67113790 2024-09-11T00:27:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.261+0000 7f08f3400640 1 -- 172.21.15.103:0/1173488951 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f800fac0 con 0x555c67113790 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.261+0000 7f08f3400640 1 -- 172.21.15.103:0/1173488951 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8004640 con 0x555c67113790 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 -- 172.21.15.103:0/1173488951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 msgr2=0x555c6711c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711c060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f08f801c910 tx=0x7f08f800f830 comp rx=0 tx=0).stop 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 -- 172.21.15.103:0/1173488951 shutdown_connections 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c67118ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c671160c0 0x555c6711c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 --2- 172.21.15.103:0/1173488951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c67113790 0x555c6711c060 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 -- 172.21.15.103:0/1173488951 >> 172.21.15.103:0/1173488951 conn(0x555c6710e8e0 msgr2=0x555c67112d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 -- 172.21.15.103:0/1173488951 shutdown_connections 2024-09-11T00:27:42.266 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 -- 172.21.15.103:0/1173488951 wait complete. 2024-09-11T00:27:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.262+0000 7f0910e2f740 1 -- start start 2024-09-11T00:27:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 1 Processor -- start 2024-09-11T00:27:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67113790 0x555c67124580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c671160c0 0x555c67124c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c671233c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09081abe20 con 0x555c67118b00 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f09081ad290 con 0x555c671160c0 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09081ac4f0 con 0x555c67113790 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0910e2f740 10 client.0 _ll_register_callbacks cb 0x7f08f8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0907400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67113790 0x555c67124580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0907400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67113790 0x555c67124580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:36888/0 (socket says 172.21.15.103:36888) 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0907400640 1 -- 172.21.15.103:0/2360585090 learned_addr learned my addr 172.21.15.103:0/2360585090 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0906a00640 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c671160c0 0x555c67124c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.263+0000 7f0907e00640 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c671233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f0907400640 1 -- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c671160c0 msgr2=0x555c67124c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:42.269 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f0907400640 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c671160c0 0x555c67124c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f0907400640 1 -- 172.21.15.103:0/2360585090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 msgr2=0x555c671233c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f0907400640 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c671233c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f0907400640 1 -- 172.21.15.103:0/2360585090 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c67123bf0 con 0x555c67113790 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f0907400640 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67113790 0x555c67124580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009530 tx=0x7f08f801ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b36a57534780691 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f08f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f08f0c00640 1 -- 172.21.15.103:0/2360585090 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8039060 con 0x555c67113790 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08f800fac0 con 0x555c67113790 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.264+0000 7f08f0c00640 1 -- 172.21.15.103:0/2360585090 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f08f8018040 con 0x555c67113790 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f08f0c00640 1 -- 172.21.15.103:0/2360585090 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f08f8040050 con 0x555c67113790 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 10 client.25891 fetch_fsmap learned FSMap version 37 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555c671229d0 con 0x555c67113790 2024-09-11T00:27:42.270 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f08f0c00640 1 -- 172.21.15.103:0/2360585090 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f800fac0 con 0x555c67113790 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f08f0c00640 1 -- 172.21.15.103:0/2360585090 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7f08f8003c30 con 0x555c67113790 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 10 client.25891 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 10 client.25891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 20 client.25891 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c67197e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 4 client.25891 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c670840b0 con 0x555c67128930 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.265+0000 7f0906a00640 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c67197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.266+0000 7f0906a00640 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c67197e70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70762a77f9e56a4a server_cookie=78385299928e306e in_seq=0 out_seq=0 2024-09-11T00:27:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.266+0000 7f08f0c00640 10 client.25891 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f08f0c00640 1 -- 172.21.15.103:0/2360585090 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c670840b0 con 0x555c67128930 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f08f0c00640 10 client.25891 handle_command_reply: tid=0 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 client.25891 shutdown 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67113790 msgr2=0x555c67124580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67113790 0x555c67124580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009530 tx=0x7f08f801ccb0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 shutdown_connections 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555c67118b00 0x555c671233c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x555c671160c0 0x555c67124c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x555c67128930 0x555c67197e70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 --2- 172.21.15.103:0/2360585090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555c67113790 0x555c67124580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f08f0c00640 0 client.25891 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.281+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 >> 172.21.15.103:0/2360585090 conn(0x555c6710e8e0 msgr2=0x555c67112cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 shutdown_connections 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f0910e2f740 1 -- 172.21.15.103:0/2360585090 wait complete. 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f0910e2f740 20 client.25891 trim_cache size 0 max 16384 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:27:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:27:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:27:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f08ec0b52e0 msgr2=0x7f08ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f090e000640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f08ec0b52e0 0x7f08ec0b77a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0900009960 tx=0x7f090003d040 comp rx=0 tx=0).stop 2024-09-11T00:27:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908105cf0 msgr2=0x7f090819b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f090e000640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908105cf0 0x7f090819b560 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f08f400e9f0 tx=0x7f08f400eeb0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 shutdown_connections 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f090e000640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f08ec0b52e0 0x7f08ec0b77a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.282+0000 7f090e000640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09081066b0 0x7f090819c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.283+0000 7f090e000640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0908105cf0 0x7f090819b560 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.283+0000 7f090e000640 1 --2- 172.21.15.103:0/3140660398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0908104ba0 0x7f090819af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.283+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 >> 172.21.15.103:0/3140660398 conn(0x7f0908100460 msgr2=0x7f0908109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.283+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 shutdown_connections 2024-09-11T00:27:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:42.283+0000 7f090e000640 1 -- 172.21.15.103:0/3140660398 wait complete. 2024-09-11T00:27:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:42 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:27:42.821 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:42 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:27:42.838 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:42 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:27:43.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:43 smithi183 ceph-mon[35934]: pgmap v539: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 239 op/s 2024-09-11T00:27:43.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:27:43.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:27:43.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:27:43.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:43 smithi103 ceph-mon[30578]: pgmap v539: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 239 op/s 2024-09-11T00:27:43.845 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:27:43.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:27:43.846 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:27:43.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:43 smithi117 ceph-mon[35525]: pgmap v539: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 239 op/s 2024-09-11T00:27:43.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:27:43.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:27:43.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:27:45.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:45 smithi103 ceph-mon[30578]: pgmap v540: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-09-11T00:27:45.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:45 smithi183 ceph-mon[35934]: pgmap v540: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-09-11T00:27:45.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:45 smithi117 ceph-mon[35525]: pgmap v540: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 244 op/s 2024-09-11T00:27:47.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:47 smithi103 ceph-mon[30578]: pgmap v541: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 488 KiB/s wr, 104 op/s 2024-09-11T00:27:47.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:47 smithi117 ceph-mon[35525]: pgmap v541: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 488 KiB/s wr, 104 op/s 2024-09-11T00:27:48.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:47 smithi183 ceph-mon[35934]: pgmap v541: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 488 KiB/s wr, 104 op/s 2024-09-11T00:27:49.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:49 smithi103 ceph-mon[30578]: pgmap v542: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 488 KiB/s wr, 105 op/s 2024-09-11T00:27:49.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:49 smithi117 ceph-mon[35525]: pgmap v542: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 488 KiB/s wr, 105 op/s 2024-09-11T00:27:50.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:49 smithi183 ceph-mon[35934]: pgmap v542: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 488 KiB/s wr, 105 op/s 2024-09-11T00:27:51.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:51 smithi103 ceph-mon[30578]: pgmap v543: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 270 KiB/s wr, 52 op/s 2024-09-11T00:27:51.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:51 smithi117 ceph-mon[35525]: pgmap v543: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 270 KiB/s wr, 52 op/s 2024-09-11T00:27:52.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:51 smithi183 ceph-mon[35934]: pgmap v543: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 270 KiB/s wr, 52 op/s 2024-09-11T00:27:53.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:53 smithi103 ceph-mon[30578]: pgmap v544: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-11T00:27:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:53 smithi117 ceph-mon[35525]: pgmap v544: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-11T00:27:54.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:53 smithi183 ceph-mon[35934]: pgmap v544: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3299385492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e88102990 msgr2=0x7f4e88102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3299385492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e88102990 0x7f4e88102d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009930 tx=0x7f4e7c02a420 comp rx=0 tx=0).stop 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3299385492 shutdown_connections 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3299385492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e881044a0 0x7f4e8810ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3299385492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e88103ae0 0x7f4e88103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3299385492 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e88102990 0x7f4e88102d90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3299385492 >> 172.21.15.103:0/3299385492 conn(0x7f4e880fe250 msgr2=0x7f4e88100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.415+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3299385492 shutdown_connections 2024-09-11T00:27:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.416+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3299385492 wait complete. 2024-09-11T00:27:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.416+0000 7f4e8fe00640 1 Processor -- start 2024-09-11T00:27:55.420 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8fe00640 1 -- start start 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8fe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e88102990 0x7f4e8819afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e88103ae0 0x7f4e8819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e881044a0 0x7f4e8819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8810e650 con 0x7f4e881044a0 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8fe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8810e4d0 con 0x7f4e88102990 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8fe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8810e7d0 con 0x7f4e88103ae0 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e881044a0 0x7f4e8819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e881044a0 0x7f4e8819c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42464/0 (socket says 172.21.15.103:42464) 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8de00640 1 -- 172.21.15.103:0/3663299876 learned_addr learned my addr 172.21.15.103:0/3663299876 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e88103ae0 0x7f4e8819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.417+0000 7f4e8d400640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e88102990 0x7f4e8819afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8d400640 1 -- 172.21.15.103:0/3663299876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e88103ae0 msgr2=0x7f4e8819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8d400640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e88103ae0 0x7f4e8819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8d400640 1 -- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e881044a0 msgr2=0x7f4e8819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8d400640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e881044a0 0x7f4e8819c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8d400640 1 -- 172.21.15.103:0/3663299876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8819cb10 con 0x7f4e88102990 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8de00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e881044a0 0x7f4e8819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8d400640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e88102990 0x7f4e8819afd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c02a940 tx=0x7f4e7c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e86000640 1 -- 172.21.15.103:0/3663299876 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c03f070 con 0x7f4e88102990 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8819cda0 con 0x7f4e88102990 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e86000640 1 -- 172.21.15.103:0/3663299876 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7c031c50 con 0x7f4e88102990 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e86000640 1 -- 172.21.15.103:0/3663299876 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c035030 con 0x7f4e88102990 2024-09-11T00:27:55.422 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.418+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e881a34d0 con 0x7f4e88102990 2024-09-11T00:27:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.421+0000 7f4e92dc2740 1 Processor -- start 2024-09-11T00:27:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e92dc2740 1 -- start start 2024-09-11T00:27:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddf5790 0x55872ddf7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 0x55872ddfa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddfab00 0x55872ddfcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55872dd686b0 con 0x55872ddf5790 2024-09-11T00:27:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55872dd57ae0 con 0x55872ddfab00 2024-09-11T00:27:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55872dd5b8f0 con 0x55872ddf80c0 2024-09-11T00:27:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddf5790 0x55872ddf7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddf5790 0x55872ddf7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42476/0 (socket says 172.21.15.103:42476) 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e8de00640 1 -- 172.21.15.103:0/3500586175 learned_addr learned my addr 172.21.15.103:0/3500586175 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e86000640 1 -- 172.21.15.103:0/3663299876 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4e7c0317a0 con 0x7f4e88102990 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.422+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddfab00 0x55872ddfcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e8de00640 1 -- 172.21.15.103:0/3500586175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 msgr2=0x55872ddfa4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e8de00640 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 0x55872ddfa4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e8de00640 1 -- 172.21.15.103:0/3500586175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddfab00 msgr2=0x55872ddfcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e8de00640 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddfab00 0x55872ddfcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e8de00640 1 -- 172.21.15.103:0/3500586175 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872dd24780 con 0x55872ddf5790 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddfab00 0x55872ddfcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e8de00640 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddf5790 0x55872ddf7b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4e80011890 tx=0x7f4e80011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f9dff64bd63b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e7a600640 1 -- 172.21.15.103:0/3500586175 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80011310 con 0x55872ddf5790 2024-09-11T00:27:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e7a600640 1 -- 172.21.15.103:0/3500586175 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8000f9a0 con 0x55872ddf5790 2024-09-11T00:27:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.423+0000 7f4e7a600640 1 -- 172.21.15.103:0/3500586175 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80005650 con 0x55872ddf5790 2024-09-11T00:27:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 -- 172.21.15.103:0/3500586175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddf5790 msgr2=0x55872ddf7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddf5790 0x55872ddf7b80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4e80011890 tx=0x7f4e80011d50 comp rx=0 tx=0).stop 2024-09-11T00:27:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 -- 172.21.15.103:0/3500586175 shutdown_connections 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddfab00 0x55872ddfcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 0x55872ddfa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/3500586175 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddf5790 0x55872ddf7b80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 -- 172.21.15.103:0/3500586175 >> 172.21.15.103:0/3500586175 conn(0x55872ddf08e0 msgr2=0x55872ddf1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e86000640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b52e0 0x7f4e700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 -- 172.21.15.103:0/3500586175 shutdown_connections 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e92dc2740 1 -- 172.21.15.103:0/3500586175 wait complete. 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e86000640 1 -- 172.21.15.103:0/3663299876 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f4e7c1068e0 con 0x7f4e88102990 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.424+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b52e0 0x7f4e700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 -- start start 2024-09-11T00:27:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 Processor -- start 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b52e0 0x7f4e700b77a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009c10 tx=0x7f4e74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872ddfee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872ddff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de00060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55872dc8fab0 con 0x55872ddfab00 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55872dcfe570 con 0x55872ddf80c0 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55872ddfdae0 con 0x55872ddf5790 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.425+0000 7f4e92dc2740 10 client.0 _ll_register_callbacks cb 0x55872dd0d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872ddfee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de00060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872ddff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872ddfee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44846/0 (socket says 172.21.15.103:44846) 2024-09-11T00:27:55.436 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 -- 172.21.15.103:0/784986884 learned_addr learned my addr 172.21.15.103:0/784986884 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 -- 172.21.15.103:0/784986884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 msgr2=0x55872ddff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872ddff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 -- 172.21.15.103:0/784986884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 msgr2=0x55872de00060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de00060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 -- 172.21.15.103:0/784986884 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55872dab9790 con 0x55872ddf5790 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e8d400640 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872ddfee60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0c9fd0 tx=0x7f4e7c02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ce3a76ab8832ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e6ea00640 1 -- 172.21.15.103:0/784986884 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c049040 con 0x55872ddf5790 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e8000f9a0 con 0x55872ddf5790 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.426+0000 7f4e6ea00640 1 -- 172.21.15.103:0/784986884 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f4e7c0c8fd0 con 0x55872ddf5790 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e6ea00640 1 -- 172.21.15.103:0/784986884 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4e7c0cc6b0 con 0x55872ddf5790 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 10 client.25900 fetch_fsmap learned FSMap version 37 2024-09-11T00:27:55.437 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55872db32620 con 0x55872ddf5790 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e6ea00640 1 -- 172.21.15.103:0/784986884 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8000f9a0 con 0x55872ddf5790 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e6ea00640 1 -- 172.21.15.103:0/784986884 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7f4e7c0d5db0 con 0x55872ddf5790 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 10 client.25900 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 10 client.25900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 20 client.25900 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de0a670 0x55872de0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 4 client.25900 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.427+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55872dd6a600 con 0x55872de0a670 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.428+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de0a670 0x55872de0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.428+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de0a670 0x55872de0ca90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11681713cbe63b4a server_cookie=6799970c0ab27ffb in_seq=0 out_seq=0 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.428+0000 7f4e6ea00640 10 client.25900 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.429+0000 7f4e6ea00640 1 -- 172.21.15.103:0/784986884 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55872dd6a600 con 0x55872de0a670 2024-09-11T00:27:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.429+0000 7f4e6ea00640 10 client.25900 handle_command_reply: tid=0 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.429+0000 7f4e92dc2740 1 client.25900 shutdown 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.429+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 msgr2=0x55872ddfee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.429+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872ddfee60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0c9fd0 tx=0x7f4e7c02a420 comp rx=0 tx=0).stop 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 shutdown_connections 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de00060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872ddff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de0a670 0x55872de0ca90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/784986884 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872ddfee60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 >> 172.21.15.103:0/784986884 conn(0x55872ddf08e0 msgr2=0x55872ddefa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e6ea00640 0 client.25900 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 shutdown_connections 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 1 -- 172.21.15.103:0/784986884 wait complete. 2024-09-11T00:27:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.430+0000 7f4e92dc2740 20 client.25900 trim_cache size 0 max 16384 2024-09-11T00:27:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 Processor -- start 2024-09-11T00:27:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 -- start start 2024-09-11T00:27:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf5790 0x55872ddeb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 0x55872ddf4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872ddf5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881b01e0 con 0x55872ddfab00 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881ad9f0 con 0x55872ddf5790 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881ab840 con 0x55872ddf80c0 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872ddf5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e8de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872ddf5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42492/0 (socket says 172.21.15.103:42492) 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e8de00640 1 -- 172.21.15.103:0/1916595946 learned_addr learned my addr 172.21.15.103:0/1916595946 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.445+0000 7f4e8d400640 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf5790 0x55872ddeb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 0x55872ddf4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e8de00640 1 -- 172.21.15.103:0/1916595946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 msgr2=0x55872ddf4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e8de00640 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 0x55872ddf4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e8de00640 1 -- 172.21.15.103:0/1916595946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf5790 msgr2=0x55872ddeb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e8de00640 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf5790 0x55872ddeb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e8de00640 1 -- 172.21.15.103:0/1916595946 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55872ddfe960 con 0x55872ddfab00 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e8de00640 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872ddf5100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009aa0 tx=0x7f4e80011110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e949c16c2dad89 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e79c00640 1 -- 172.21.15.103:0/1916595946 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8001e040 con 0x55872ddfab00 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e79c00640 1 -- 172.21.15.103:0/1916595946 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8000f9a0 con 0x55872ddfab00 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.446+0000 7f4e79c00640 1 -- 172.21.15.103:0/1916595946 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80005460 con 0x55872ddfab00 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 -- 172.21.15.103:0/1916595946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 msgr2=0x55872ddf5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872ddf5100 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009aa0 tx=0x7f4e80011110 comp rx=0 tx=0).stop 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 -- 172.21.15.103:0/1916595946 shutdown_connections 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872ddf5100 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf80c0 0x55872ddf4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/1916595946 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf5790 0x55872ddeb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 -- 172.21.15.103:0/1916595946 >> 172.21.15.103:0/1916595946 conn(0x55872ddf08e0 msgr2=0x55872ddf3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 -- 172.21.15.103:0/1916595946 shutdown_connections 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 -- 172.21.15.103:0/1916595946 wait complete. 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.447+0000 7f4e92dc2740 1 -- start start 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 1 Processor -- start 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872de06060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872de065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de04930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881abeb0 con 0x55872ddfab00 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881ad320 con 0x55872ddf80c0 2024-09-11T00:27:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e881ac580 con 0x55872ddf5790 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e92dc2740 10 client.0 _ll_register_callbacks cb 0x7f4e7c031ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e8ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872de065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e8ca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872de065a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41056/0 (socket says 172.21.15.103:41056) 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e8ca00640 1 -- 172.21.15.103:0/846698039 learned_addr learned my addr 172.21.15.103:0/846698039 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e8de00640 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de04930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.448+0000 7f4e8d400640 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872de06060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e8ca00640 1 -- 172.21.15.103:0/846698039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 msgr2=0x55872de06060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872de06060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e8ca00640 1 -- 172.21.15.103:0/846698039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 msgr2=0x55872de04930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de04930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e8ca00640 1 -- 172.21.15.103:0/846698039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55872de0d110 con 0x55872ddf80c0 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e8ca00640 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872de065a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55872de0d310 tx=0x7f4e740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c34f6874176f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e6ea00640 1 -- 172.21.15.103:0/846698039 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74036060 con 0x55872ddf80c0 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e8000f9a0 con 0x55872ddf80c0 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e6ea00640 1 -- 172.21.15.103:0/846698039 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f4e74003d90 con 0x55872ddf80c0 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e6ea00640 1 -- 172.21.15.103:0/846698039 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4e7403d050 con 0x55872ddf80c0 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e92dc2740 10 client.25880 fetch_fsmap learned FSMap version 37 2024-09-11T00:27:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.449+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55872de77e50 con 0x55872ddf80c0 2024-09-11T00:27:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e6ea00640 1 -- 172.21.15.103:0/846698039 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8000f9a0 con 0x55872ddf80c0 2024-09-11T00:27:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e6ea00640 1 -- 172.21.15.103:0/846698039 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 37) ==== 8378+0+0 (secure 0 0 0) 0x7f4e7401f4b0 con 0x55872ddf80c0 2024-09-11T00:27:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e92dc2740 10 client.25880 fetch_fsmap finished waiting for FSMap version 37 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e92dc2740 10 client.25880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e92dc2740 20 client.25880 populate_metadata read hostname 'smithi103' 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de78840 0x55872de7ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e92dc2740 4 client.25880 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55872dd660b0 con 0x55872de78840 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.450+0000 7f4e8d400640 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de78840 0x55872de7ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.451+0000 7f4e8d400640 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de78840 0x55872de7ac60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10c1ca2002a8356 server_cookie=7c54240e7ec79f77 in_seq=0 out_seq=0 2024-09-11T00:27:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.451+0000 7f4e6ea00640 10 client.25880 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e6ea00640 1 -- 172.21.15.103:0/846698039 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55872dd660b0 con 0x55872de78840 2024-09-11T00:27:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e6ea00640 10 client.25880 handle_command_reply: tid=0 2024-09-11T00:27:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 client.25880 shutdown 2024-09-11T00:27:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 msgr2=0x55872de065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872de065a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55872de0d310 tx=0x7f4e740046c0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 shutdown_connections 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55872ddfab00 0x55872de04930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55872ddf80c0 0x55872de065a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55872de78840 0x55872de7ac60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 --2- 172.21.15.103:0/846698039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55872ddf5790 0x55872de06060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e6ea00640 0 client.25880 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 >> 172.21.15.103:0/846698039 conn(0x55872ddf08e0 msgr2=0x55872ddf4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 shutdown_connections 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 1 -- 172.21.15.103:0/846698039 wait complete. 2024-09-11T00:27:55.465 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.462+0000 7f4e92dc2740 20 client.25880 trim_cache size 0 max 16384 2024-09-11T00:27:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:27:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:27:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:27:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b52e0 msgr2=0x7f4e700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b52e0 0x7f4e700b77a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e74009c10 tx=0x7f4e74005f90 comp rx=0 tx=0).stop 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e88102990 msgr2=0x7f4e8819afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e88102990 0x7f4e8819afd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c02a940 tx=0x7f4e7c003c10 comp rx=0 tx=0).stop 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 shutdown_connections 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e700b52e0 0x7f4e700b77a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e881044a0 0x7f4e8819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.467 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e88103ae0 0x7f4e8819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 --2- 172.21.15.103:0/3663299876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e88102990 0x7f4e8819afd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:27:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 >> 172.21.15.103:0/3663299876 conn(0x7f4e880fe250 msgr2=0x7f4e88075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:27:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 shutdown_connections 2024-09-11T00:27:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:27:55.463+0000 7f4e8fe00640 1 -- 172.21.15.103:0/3663299876 wait complete. 2024-09-11T00:27:55.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:55 smithi103 ceph-mon[30578]: pgmap v545: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-11T00:27:55.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:55 smithi117 ceph-mon[35525]: pgmap v545: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-11T00:27:56.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:55 smithi183 ceph-mon[35934]: pgmap v545: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 28 KiB/s wr, 13 op/s 2024-09-11T00:27:57.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:57 smithi103 ceph-mon[30578]: pgmap v546: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-09-11T00:27:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:57 smithi117 ceph-mon[35525]: pgmap v546: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-09-11T00:27:58.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:57 smithi183 ceph-mon[35934]: pgmap v546: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 23 KiB/s wr, 8 op/s 2024-09-11T00:27:59.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:59 smithi103 ceph-mon[30578]: pgmap v547: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-09-11T00:27:59.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:27:59 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:27:59.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:59 smithi117 ceph-mon[35525]: pgmap v547: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-09-11T00:27:59.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:27:59 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:00.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:59 smithi183 ceph-mon[35934]: pgmap v547: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-09-11T00:28:00.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:27:59 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:00.688 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:28:01.073 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:01.683 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 -- 172.21.15.103:0/19665105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec0102dc0 msgr2=0x7f4ec01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 --2- 172.21.15.103:0/19665105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec0102dc0 0x7f4ec01031a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009900 tx=0x7f4eb002a410 comp rx=0 tx=0).stop 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 -- 172.21.15.103:0/19665105 shutdown_connections 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 --2- 172.21.15.103:0/19665105 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ec0109150 0x7f4ec010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 --2- 172.21.15.103:0/19665105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec01087d0 0x7f4ec0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 --2- 172.21.15.103:0/19665105 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec0102dc0 0x7f4ec01031a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 -- 172.21.15.103:0/19665105 >> 172.21.15.103:0/19665105 conn(0x7f4ec00fe830 msgr2=0x7f4ec0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 -- 172.21.15.103:0/19665105 shutdown_connections 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.681+0000 7f4ec5000640 1 -- 172.21.15.103:0/19665105 wait complete. 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 Processor -- start 2024-09-11T00:28:01.684 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 -- start start 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0102dc0 0x7f4ec019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec01087d0 0x7f4ec019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ec0109150 0x7f4ec01a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0112e60 con 0x7f4ec01087d0 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0112ce0 con 0x7f4ec0102dc0 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.682+0000 7f4ec5000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec0112fe0 con 0x7f4ec0109150 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebd400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec01087d0 0x7f4ec019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebde00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0102dc0 0x7f4ec019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebd400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec01087d0 0x7f4ec019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42536/0 (socket says 172.21.15.103:42536) 2024-09-11T00:28:01.686 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebd400640 1 -- 172.21.15.103:0/475662372 learned_addr learned my addr 172.21.15.103:0/475662372 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebe800640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ec0109150 0x7f4ec01a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebde00640 1 -- 172.21.15.103:0/475662372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ec0109150 msgr2=0x7f4ec01a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebde00640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ec0109150 0x7f4ec01a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebde00640 1 -- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec01087d0 msgr2=0x7f4ec019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebde00640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec01087d0 0x7f4ec019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebde00640 1 -- 172.21.15.103:0/475662372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec01a1570 con 0x7f4ec0102dc0 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.683+0000 7f4ebde00640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0102dc0 0x7f4ec019b4c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002aa00 tx=0x7f4eb0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.684+0000 7f4eb6a00640 1 -- 172.21.15.103:0/475662372 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb003f070 con 0x7f4ec0102dc0 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.684+0000 7f4eb6a00640 1 -- 172.21.15.103:0/475662372 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4eb0031b50 con 0x7f4ec0102dc0 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.684+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec01a1800 con 0x7f4ec0102dc0 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.684+0000 7f4eb6a00640 1 -- 172.21.15.103:0/475662372 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0035030 con 0x7f4ec0102dc0 2024-09-11T00:28:01.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.684+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec01a7f20 con 0x7f4ec0102dc0 2024-09-11T00:28:01.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.685+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec01031a0 con 0x7f4ec0102dc0 2024-09-11T00:28:01.689 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.687+0000 7f4eb6a00640 1 -- 172.21.15.103:0/475662372 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4eb00316a0 con 0x7f4ec0102dc0 2024-09-11T00:28:01.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.688+0000 7f4eb6a00640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4ea40b5360 0x7f4ea40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:01.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.688+0000 7f4eb6a00640 1 -- 172.21.15.103:0/475662372 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f4eb01068b0 con 0x7f4ec0102dc0 2024-09-11T00:28:01.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.689+0000 7f4ebd400640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4ea40b5360 0x7f4ea40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:01.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.689+0000 7f4ebd400640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4ea40b5360 0x7f4ea40b7820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ec01081d0 tx=0x7f4eac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:01.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.690+0000 7f4eb6a00640 1 -- 172.21.15.103:0/475662372 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4eb00ccd30 con 0x7f4ec0102dc0 2024-09-11T00:28:01.910 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.908+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ec002e5c0 con 0x7f4ec0102dc0 2024-09-11T00:28:01.910 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:01 smithi103 ceph-mon[30578]: pgmap v548: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-11T00:28:01.911 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.909+0000 7f4eb6a00640 1 -- 172.21.15.103:0/475662372 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11940 (secure 0 0 0) 0x7f4eb00d1760 con 0x7f4ec0102dc0 2024-09-11T00:28:01.912 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:28:01.912 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":38,"btime":"2024-09-11T00:27:58:548218+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:27:57.684224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25858":{"gid":25858,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2940154743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2940154743},{"type":"v1","addr":"172.21.15.183:6835","nonce":2940154743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:28:01.912 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 38 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.911+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4ea40b5360 msgr2=0x7f4ea40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.911+0000 7f4ec5000640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4ea40b5360 0x7f4ea40b7820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4ec01081d0 tx=0x7f4eac005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.911+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0102dc0 msgr2=0x7f4ec019b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.911+0000 7f4ec5000640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0102dc0 0x7f4ec019b4c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002aa00 tx=0x7f4eb0003b90 comp rx=0 tx=0).stop 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 shutdown_connections 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4ea40b5360 0x7f4ea40b7820 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4ec0109150 0x7f4ec01a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ec01087d0 0x7f4ec019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 --2- 172.21.15.103:0/475662372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ec0102dc0 0x7f4ec019b4c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 >> 172.21.15.103:0/475662372 conn(0x7f4ec00fe830 msgr2=0x7f4ec0104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 shutdown_connections 2024-09-11T00:28:01.914 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:01.912+0000 7f4ec5000640 1 -- 172.21.15.103:0/475662372 wait complete. 2024-09-11T00:28:01.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:01 smithi117 ceph-mon[35525]: pgmap v548: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-11T00:28:02.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:01 smithi183 ceph-mon[35934]: pgmap v548: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 25 KiB/s wr, 10 op/s 2024-09-11T00:28:02.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:02 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/475662372' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:02.875 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --set-id 76cb9b6b --release --await' 2024-09-11T00:28:02.875 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --set-id 76cb9b6b --release --await 2024-09-11T00:28:02.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:02 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/475662372' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:03.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:02 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/475662372' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:03.254 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:03.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:03 smithi103 ceph-mon[30578]: pgmap v549: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-09-11T00:28:03.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 -- 172.21.15.103:0/3786038640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108fa0 msgr2=0x7feecc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:03.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 --2- 172.21.15.103:0/3786038640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108fa0 0x7feecc10f7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feec4009ca0 tx=0x7feec402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 -- 172.21.15.103:0/3786038640 shutdown_connections 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 --2- 172.21.15.103:0/3786038640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108fa0 0x7feecc10f7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 --2- 172.21.15.103:0/3786038640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc108620 0x7feecc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 --2- 172.21.15.103:0/3786038640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feecc102c10 0x7feecc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 -- 172.21.15.103:0/3786038640 >> 172.21.15.103:0/3786038640 conn(0x7feecc0fe6c0 msgr2=0x7feecc100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 -- 172.21.15.103:0/3786038640 shutdown_connections 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.840+0000 7feecb200640 1 -- 172.21.15.103:0/3786038640 wait complete. 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.841+0000 7feecb200640 1 Processor -- start 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feecb200640 1 -- start start 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feecb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc102c10 0x7feecc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feecb200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108620 0x7feecc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.877 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feecb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feecc108fa0 0x7feecc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feecb200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feecc112c70 con 0x7feecc108620 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feecb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feecc112af0 con 0x7feecc108fa0 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feecb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feecc112df0 con 0x7feecc102c10 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feec9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108620 0x7feecc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feec9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108620 0x7feecc0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50024/0 (socket says 172.21.15.103:50024) 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feec9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc102c10 0x7feecc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.842+0000 7feec9400640 1 -- 172.21.15.103:0/2476541387 learned_addr learned my addr 172.21.15.103:0/2476541387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feeca800640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feecc108fa0 0x7feecc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec9400640 1 -- 172.21.15.103:0/2476541387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc102c10 msgr2=0x7feecc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec9400640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc102c10 0x7feecc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec9400640 1 -- 172.21.15.103:0/2476541387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feecc108fa0 msgr2=0x7feecc075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec9400640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feecc108fa0 0x7feecc075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec9400640 1 -- 172.21.15.103:0/2476541387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc1a7400 con 0x7feecc108620 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec9400640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108620 0x7feecc0759e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feeb800e9f0 tx=0x7feeb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec2a00640 1 -- 172.21.15.103:0/2476541387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb800cd80 con 0x7feecc108620 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc1a76f0 con 0x7feecc108620 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec2a00640 1 -- 172.21.15.103:0/2476541387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb80054d0 con 0x7feecc108620 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.843+0000 7feec2a00640 1 -- 172.21.15.103:0/2476541387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8010690 con 0x7feecc108620 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.844+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc1a7c30 con 0x7feecc108620 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 Processor -- start 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 -- start start 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee87f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee88b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 0x5606ee88de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5606ee7f6290 con 0x5606ee87faf0 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5606ee7e56c0 con 0x5606ee88ba80 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feed3c8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5606ee7e94d0 con 0x5606ee87f1d0 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feec9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee88b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feeca800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee87f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feec9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee88b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50030/0 (socket says 172.21.15.103:50030) 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.847+0000 7feec9e00640 1 -- 172.21.15.103:0/91105985 learned_addr learned my addr 172.21.15.103:0/91105985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feec2a00640 1 -- 172.21.15.103:0/2476541387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7feeb8004e90 con 0x7feecc108620 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feec9e00640 1 -- 172.21.15.103:0/91105985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 msgr2=0x5606ee87f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feec9e00640 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee87f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feec9e00640 1 -- 172.21.15.103:0/91105985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 msgr2=0x5606ee88de70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feec9e00640 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 0x5606ee88de70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feec9e00640 1 -- 172.21.15.103:0/91105985 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ee772460 con 0x5606ee87faf0 2024-09-11T00:28:03.880 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feeca800640 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee87f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feec9e00640 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee88b540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feebc00bd80 tx=0x7feebc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f6a5c3641fb1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feeb7400640 1 -- 172.21.15.103:0/91105985 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc005220 con 0x5606ee87faf0 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feeb7400640 1 -- 172.21.15.103:0/91105985 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feebc004c50 con 0x5606ee87faf0 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.848+0000 7feeb7400640 1 -- 172.21.15.103:0/91105985 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc011800 con 0x5606ee87faf0 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.849+0000 7feed3c8b740 1 -- 172.21.15.103:0/91105985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 msgr2=0x5606ee88b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.849+0000 7feed3c8b740 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee88b540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7feebc00bd80 tx=0x7feebc0107a0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.849+0000 7feed3c8b740 1 -- 172.21.15.103:0/91105985 shutdown_connections 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 0x5606ee88de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee88b540 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 --2- 172.21.15.103:0/91105985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee87f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 -- 172.21.15.103:0/91105985 >> 172.21.15.103:0/91105985 conn(0x5606ee87d000 msgr2=0x5606ee87e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:03.881 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 -- 172.21.15.103:0/91105985 shutdown_connections 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 -- 172.21.15.103:0/91105985 wait complete. 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feec2a00640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feeb00b5320 0x7feeb00b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 -- start start 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feec2a00640 1 -- 172.21.15.103:0/2476541387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7feeb80e62a0 con 0x7feecc108620 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feeca800640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feeb00b5320 0x7feeb00b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.850+0000 7feed3c8b740 1 Processor -- start 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee888a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87faf0 0x5606ee888f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee88ba80 0x5606ee8894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feed3c8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5606ee646bb0 con 0x5606ee88ba80 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feed3c8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5606ee7806c0 con 0x5606ee87faf0 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feed3c8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5606ee88e730 con 0x5606ee87f1d0 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feed3c8b740 10 client.0 _ll_register_callbacks cb 0x5606ee79b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:03.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feeca800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee88ba80 0x5606ee8894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feec9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87faf0 0x5606ee888f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feec9400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87faf0 0x5606ee888f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51928/0 (socket says 172.21.15.103:51928) 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feec9400640 1 -- 172.21.15.103:0/846571440 learned_addr learned my addr 172.21.15.103:0/846571440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feec9e00640 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee888a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.851+0000 7feec9400640 1 -- 172.21.15.103:0/846571440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 msgr2=0x5606ee888a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feec9400640 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee888a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feec9400640 1 -- 172.21.15.103:0/846571440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee88ba80 msgr2=0x5606ee8894b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feec9400640 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee88ba80 0x5606ee8894b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feec9400640 1 -- 172.21.15.103:0/846571440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ee543b00 con 0x5606ee87faf0 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feec9400640 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87faf0 0x5606ee888f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feeb800cfd0 tx=0x7feeb80ac500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d474bec3a760996d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feeabe00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feeca800640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feeb00b5320 0x7feeb00b77e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feecc077560 tx=0x7feec4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:03.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feeabe00640 1 -- 172.21.15.103:0/846571440 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80acd70 con 0x5606ee87faf0 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feebc004c50 con 0x5606ee87faf0 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feeabe00640 1 -- 172.21.15.103:0/846571440 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7feeb80b5770 con 0x5606ee87faf0 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feeabe00640 1 -- 172.21.15.103:0/846571440 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feeb80ac670 con 0x5606ee87faf0 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feed3c8b740 10 client.25895 fetch_fsmap learned FSMap version 38 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.852+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5606ee5ba960 con 0x5606ee87faf0 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feeabe00640 1 -- 172.21.15.103:0/846571440 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feebc004c50 con 0x5606ee87faf0 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feeabe00640 1 -- 172.21.15.103:0/846571440 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7feeb80a7fc0 con 0x5606ee87faf0 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feed3c8b740 10 client.25895 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feed3c8b740 10 client.25895 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feed3c8b740 20 client.25895 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feed3c8b740 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee89b590 0x5606ee89d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feed3c8b740 4 client.25895 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:03.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606ee7f81e0 con 0x5606ee89b590 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.853+0000 7feec9e00640 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee89b590 0x5606ee89d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.854+0000 7feec9e00640 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee89b590 0x5606ee89d9b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb6c9fd0a338d59 server_cookie=b1a681a08e795d4f in_seq=0 out_seq=0 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.854+0000 7feeabe00640 10 client.25895 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.854+0000 7feeabe00640 1 -- 172.21.15.103:0/846571440 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606ee7f81e0 con 0x5606ee89b590 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.854+0000 7feeabe00640 10 client.25895 handle_command_reply: tid=0 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.854+0000 7feed3c8b740 1 client.25895 shutdown 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87faf0 msgr2=0x5606ee888f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87faf0 0x5606ee888f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feeb800cfd0 tx=0x7feeb80ac500 comp rx=0 tx=0).stop 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 shutdown_connections 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee88ba80 0x5606ee8894b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87faf0 0x5606ee888f70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee89b590 0x5606ee89d9b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.885 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 --2- 172.21.15.103:0/846571440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee888a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feeabe00640 0 client.25895 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 >> 172.21.15.103:0/846571440 conn(0x5606ee87d000 msgr2=0x5606ee882ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 shutdown_connections 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 1 -- 172.21.15.103:0/846571440 wait complete. 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.855+0000 7feed3c8b740 20 client.25895 trim_cache size 0 max 16384 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 Processor -- start 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 -- start start 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87f1d0 0x5606ee88a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee885c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee88ba80 0x5606ee8861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1b4440 con 0x5606ee87faf0 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1b1c80 con 0x5606ee87f1d0 2024-09-11T00:28:03.886 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feed3c8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1afad0 con 0x5606ee88ba80 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.873+0000 7feec9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee885c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee885c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50040/0 (socket says 172.21.15.103:50040) 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 -- 172.21.15.103:0/2195799261 learned_addr learned my addr 172.21.15.103:0/2195799261 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9e00640 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87f1d0 0x5606ee88a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feeca800640 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee88ba80 0x5606ee8861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 -- 172.21.15.103:0/2195799261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee88ba80 msgr2=0x5606ee8861a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee88ba80 0x5606ee8861a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 -- 172.21.15.103:0/2195799261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87f1d0 msgr2=0x5606ee88a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87f1d0 0x5606ee88a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 -- 172.21.15.103:0/2195799261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606ee87da80 con 0x5606ee87faf0 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feec9400640 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee885c60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c6a0 tx=0x7feeb80ac570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae3b3f88aebf5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feeb6a00640 1 -- 172.21.15.103:0/2195799261 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80a8510 con 0x5606ee87faf0 2024-09-11T00:28:03.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feeb6a00640 1 -- 172.21.15.103:0/2195799261 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb80acd70 con 0x5606ee87faf0 2024-09-11T00:28:03.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.874+0000 7feeb6a00640 1 -- 172.21.15.103:0/2195799261 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80abde0 con 0x5606ee87faf0 2024-09-11T00:28:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 -- 172.21.15.103:0/2195799261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 msgr2=0x5606ee885c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee885c60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feeb800c6a0 tx=0x7feeb80ac570 comp rx=0 tx=0).stop 2024-09-11T00:28:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 -- 172.21.15.103:0/2195799261 shutdown_connections 2024-09-11T00:28:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee88ba80 0x5606ee8861a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee885c60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.890 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 --2- 172.21.15.103:0/2195799261 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee87f1d0 0x5606ee88a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 -- 172.21.15.103:0/2195799261 >> 172.21.15.103:0/2195799261 conn(0x5606ee87d000 msgr2=0x5606ee88a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 -- 172.21.15.103:0/2195799261 shutdown_connections 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.875+0000 7feed3c8b740 1 -- 172.21.15.103:0/2195799261 wait complete. 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 -- start start 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 Processor -- start 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee894250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee890730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 0x5606ee890d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1b0140 con 0x5606ee87faf0 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1b15b0 con 0x5606ee88ba80 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1b0810 con 0x5606ee87f1d0 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.876+0000 7feed3c8b740 10 client.0 _ll_register_callbacks cb 0x7feeb800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:03.891 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee890730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee894250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee890730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50052/0 (socket says 172.21.15.103:50052) 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 -- 172.21.15.103:0/4009309789 learned_addr learned my addr 172.21.15.103:0/4009309789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee894250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:38846/0 (socket says 172.21.15.103:38846) 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feeca800640 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 0x5606ee890d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 -- 172.21.15.103:0/4009309789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 msgr2=0x5606ee894250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee894250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 -- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 msgr2=0x5606ee890d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 0x5606ee890d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 -- 172.21.15.103:0/4009309789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606ee908470 con 0x5606ee87faf0 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feec9400640 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee890730 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feeb80a8d60 tx=0x7feeb80a83b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43447e897ec2cdd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:03.892 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.877+0000 7feeabe00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feeabe00640 1 -- 172.21.15.103:0/4009309789 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80cc060 con 0x5606ee87faf0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb80acd70 con 0x5606ee87faf0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feeabe00640 1 -- 172.21.15.103:0/4009309789 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7feeb80ab340 con 0x5606ee87faf0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feeabe00640 1 -- 172.21.15.103:0/4009309789 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7feeb80b1150 con 0x5606ee87faf0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 10 client.15867 fetch_fsmap learned FSMap version 38 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5606ee9095c0 con 0x5606ee87faf0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feeabe00640 1 -- 172.21.15.103:0/4009309789 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeb80b5e50 con 0x5606ee87faf0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feeabe00640 1 -- 172.21.15.103:0/4009309789 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 38) ==== 8374+0+0 (secure 0 0 0) 0x7feeb80b22e0 con 0x5606ee87faf0 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 10 client.15867 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 10 client.15867 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 20 client.15867 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee909aa0 0x5606ee90bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:03.893 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 4 client.15867 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "76cb9b6b", "release": true, "await": true}] 2024-09-11T00:28:03.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.878+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "set_id": "76cb9b6b", "release": true, "await": true}) -- 0x5606ee7f3c90 con 0x5606ee909aa0 2024-09-11T00:28:03.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.879+0000 7feec9e00640 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee909aa0 0x5606ee90bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:03.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.879+0000 7feec9e00640 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee909aa0 0x5606ee90bec0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8c20f6c3600cc2a server_cookie=de5c62500d071159 in_seq=0 out_seq=0 2024-09-11T00:28:03.894 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:03.879+0000 7feeabe00640 10 client.15867 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:03 smithi117 ceph-mon[35525]: pgmap v549: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-09-11T00:28:04.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:03 smithi183 ceph-mon[35934]: pgmap v549: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-09-11T00:28:04.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feeabe00640 1 -- 172.21.15.103:0/4009309789 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5606ee7f3c90 con 0x5606ee909aa0 2024-09-11T00:28:04.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feeabe00640 10 client.15867 handle_command_reply: tid=0 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 client.15867 shutdown 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 msgr2=0x5606ee890730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee890730 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7feeb80a8d60 tx=0x7feeb80a83b0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 shutdown_connections 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5606ee88ba80 0x5606ee890d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606ee87faf0 0x5606ee890730 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5606ee909aa0 0x5606ee90bec0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 --2- 172.21.15.103:0/4009309789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5606ee87f1d0 0x5606ee894250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 >> 172.21.15.103:0/4009309789 conn(0x5606ee87d000 msgr2=0x5606ee88a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.072+0000 7feeabe00640 0 client.15867 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 38, 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 16, 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:28:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "76cb9b6b": { 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 16, 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:28:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:28:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:04.078 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.073+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 shutdown_connections 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.073+0000 7feed3c8b740 1 -- 172.21.15.103:0/4009309789 wait complete. 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.073+0000 7feed3c8b740 20 client.15867 trim_cache size 0 max 16384 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feeb00b5320 msgr2=0x7feeb00b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feeb00b5320 0x7feeb00b77e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7feecc077560 tx=0x7feec4005d80 comp rx=0 tx=0).stop 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108620 msgr2=0x7feecc0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108620 0x7feecc0759e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feeb800e9f0 tx=0x7feeb800edf0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 shutdown_connections 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7feeb00b5320 0x7feeb00b77e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feecc108fa0 0x7feecc075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feecc108620 0x7feecc0759e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 --2- 172.21.15.103:0/2476541387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feecc102c10 0x7feecc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:04.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 >> 172.21.15.103:0/2476541387 conn(0x7feecc0fe6c0 msgr2=0x7feecc100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:04.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.074+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 shutdown_connections 2024-09-11T00:28:04.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:04.075+0000 7feecb200640 1 -- 172.21.15.103:0/2476541387 wait complete. 2024-09-11T00:28:04.971 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '76cb9b6b', total seconds elapsed: 34.3 2024-09-11T00:28:04.971 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T00:28:04.971 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:28:05.433 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:05.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:05 smithi103 ceph-mon[30578]: pgmap v550: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-11T00:28:05.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:05 smithi117 ceph-mon[35525]: pgmap v550: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-11T00:28:06.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:05 smithi183 ceph-mon[35934]: pgmap v550: 129 pgs: 129 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-11T00:28:06.197 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0x9fdd84 2024-09-11T00:28:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.338+0000 7fda51400640 1 -- 172.21.15.103:0/4136235709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c101af0 msgr2=0x7fda4c101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.338+0000 7fda51400640 1 --2- 172.21.15.103:0/4136235709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c101af0 0x7fda4c101ed0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda38009900 tx=0x7fda3802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.339+0000 7fda51400640 1 -- 172.21.15.103:0/4136235709 shutdown_connections 2024-09-11T00:28:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.339+0000 7fda51400640 1 --2- 172.21.15.103:0/4136235709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda4c107e80 0x7fda4c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.339+0000 7fda51400640 1 --2- 172.21.15.103:0/4136235709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda4c107500 0x7fda4c107940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.339+0000 7fda51400640 1 --2- 172.21.15.103:0/4136235709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c101af0 0x7fda4c101ed0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.339+0000 7fda51400640 1 -- 172.21.15.103:0/4136235709 >> 172.21.15.103:0/4136235709 conn(0x7fda4c0fd560 msgr2=0x7fda4c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.339+0000 7fda51400640 1 -- 172.21.15.103:0/4136235709 shutdown_connections 2024-09-11T00:28:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.339+0000 7fda51400640 1 -- 172.21.15.103:0/4136235709 wait complete. 2024-09-11T00:28:06.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 Processor -- start 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 -- start start 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda4c101af0 0x7fda4c19f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c107500 0x7fda4c19ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda4c107e80 0x7fda4c1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c111b90 con 0x7fda4c107e80 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c111a10 con 0x7fda4c107500 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.340+0000 7fda51400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fda4c111d10 con 0x7fda4c101af0 2024-09-11T00:28:06.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c107500 0x7fda4c19ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c107500 0x7fda4c19ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51970/0 (socket says 172.21.15.103:51970) 2024-09-11T00:28:06.344 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 -- 172.21.15.103:0/2709884179 learned_addr learned my addr 172.21.15.103:0/2709884179 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4be00640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda4c101af0 0x7fda4c19f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda50a00640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda4c107e80 0x7fda4c1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 -- 172.21.15.103:0/2709884179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda4c101af0 msgr2=0x7fda4c19f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda4c101af0 0x7fda4c19f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 -- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda4c107e80 msgr2=0x7fda4c1a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda4c107e80 0x7fda4c1a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.341+0000 7fda4b400640 1 -- 172.21.15.103:0/2709884179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4c1a55f0 con 0x7fda4c107500 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.342+0000 7fda4b400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c107500 0x7fda4c19ff20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c920 tx=0x7fda3c00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.342+0000 7fda4be00640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda4c101af0 0x7fda4c19f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.342+0000 7fda48c00640 1 -- 172.21.15.103:0/2709884179 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c007c70 con 0x7fda4c107500 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.342+0000 7fda48c00640 1 -- 172.21.15.103:0/2709884179 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda3c007e10 con 0x7fda4c107500 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.342+0000 7fda48c00640 1 -- 172.21.15.103:0/2709884179 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00e7a0 con 0x7fda4c107500 2024-09-11T00:28:06.345 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.342+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4c1a58e0 con 0x7fda4c107500 2024-09-11T00:28:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.342+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda4c1abfa0 con 0x7fda4c107500 2024-09-11T00:28:06.346 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.343+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda4c101ed0 con 0x7fda4c107500 2024-09-11T00:28:06.347 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.345+0000 7fda48c00640 1 -- 172.21.15.103:0/2709884179 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fda3c00f450 con 0x7fda4c107500 2024-09-11T00:28:06.348 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.347+0000 7fda48c00640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda300b5460 0x7fda300b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.347+0000 7fda48c00640 1 -- 172.21.15.103:0/2709884179 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fda3c0e6210 con 0x7fda4c107500 2024-09-11T00:28:06.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.347+0000 7fda4be00640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda300b5460 0x7fda300b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:06.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.348+0000 7fda4be00640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda300b5460 0x7fda300b7920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda38004300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:06.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.351+0000 7fda48c00640 1 -- 172.21.15.103:0/2709884179 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fda3c01d060 con 0x7fda4c107500 2024-09-11T00:28:06.652 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.650+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda4c1ac3d0 con 0x7fda4c107500 2024-09-11T00:28:06.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.651+0000 7fda48c00640 1 -- 172.21.15.103:0/2709884179 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11940 (secure 0 0 0) 0x7fda3c014030 con 0x7fda4c107500 2024-09-11T00:28:06.654 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:28:06.654 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":38,"btime":"2024-09-11T00:27:58:548218+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:27:57.684224+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25858":{"gid":25858,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2940154743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2940154743},{"type":"v1","addr":"172.21.15.183:6835","nonce":2940154743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:28:06.654 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 38 2024-09-11T00:28:06.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda300b5460 msgr2=0x7fda300b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda300b5460 0x7fda300b7920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fda38000c00 tx=0x7fda38004300 comp rx=0 tx=0).stop 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c107500 msgr2=0x7fda4c19ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c107500 0x7fda4c19ff20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00c920 tx=0x7fda3c00cde0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 shutdown_connections 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fda300b5460 0x7fda300b7920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda4c107e80 0x7fda4c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda4c107500 0x7fda4c19ff20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 --2- 172.21.15.103:0/2709884179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fda4c101af0 0x7fda4c19f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.655+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 >> 172.21.15.103:0/2709884179 conn(0x7fda4c0fd560 msgr2=0x7fda4c06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.656+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 shutdown_connections 2024-09-11T00:28:06.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:06.656+0000 7fda51400640 1 -- 172.21.15.103:0/2709884179 wait complete. 2024-09-11T00:28:07.644 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T00:28:07.644 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T00:28:07.706 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:07 smithi103 ceph-mon[30578]: pgmap v551: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-09-11T00:28:07.706 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:07 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2709884179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:07 smithi117 ceph-mon[35525]: pgmap v551: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-09-11T00:28:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:07 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2709884179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:08.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:07 smithi183 ceph-mon[35934]: pgmap v551: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-09-11T00:28:08.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:07 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2709884179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:08.118 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:08.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 -- 172.21.15.103:0/2469881537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08109230 msgr2=0x7f8a08109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2469881537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08109230 0x7f8a08109630 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009900 tx=0x7f89f802a410 comp rx=0 tx=0).stop 2024-09-11T00:28:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 -- 172.21.15.103:0/2469881537 shutdown_connections 2024-09-11T00:28:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2469881537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a0810aac0 0x7f8a08111420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2469881537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a0810a100 0x7f8a0810a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2469881537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08109230 0x7f8a08109630 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 -- 172.21.15.103:0/2469881537 >> 172.21.15.103:0/2469881537 conn(0x7f8a080783d0 msgr2=0x7f8a0807a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 -- 172.21.15.103:0/2469881537 shutdown_connections 2024-09-11T00:28:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.603+0000 7f8a0f600640 1 -- 172.21.15.103:0/2469881537 wait complete. 2024-09-11T00:28:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.604+0000 7f8a0f600640 1 Processor -- start 2024-09-11T00:28:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.604+0000 7f8a0f600640 1 -- start start 2024-09-11T00:28:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.604+0000 7f8a0f600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08109230 0x7f8a0819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.604+0000 7f8a0f600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0810a100 0x7f8a0819f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.604+0000 7f8a0f600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a0810aac0 0x7f8a081a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f8a0f600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08114c40 con 0x7f8a0810aac0 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f8a0f600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08114ac0 con 0x7f8a0810a100 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f8a0f600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08114dc0 con 0x7f8a08109230 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f8a0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08109230 0x7f8a0819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0810a100 0x7f8a0819f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0810a100 0x7f8a0819f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51982/0 (socket says 172.21.15.103:51982) 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f8a0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08109230 0x7f8a0819f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:38880/0 (socket says 172.21.15.103:38880) 2024-09-11T00:28:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 -- 172.21.15.103:0/2752285727 learned_addr learned my addr 172.21.15.103:0/2752285727 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f8a0d600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a0810aac0 0x7f8a081a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 -- 172.21.15.103:0/2752285727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08109230 msgr2=0x7f8a0819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08109230 0x7f8a0819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 -- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a0810aac0 msgr2=0x7f8a081a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a0810aac0 0x7f8a081a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 -- 172.21.15.103:0/2752285727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a081a0ea0 con 0x7f8a0810a100 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.605+0000 7f89ffe00640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0810a100 0x7f8a0819f9b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89f000e9f0 tx=0x7f89f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.606+0000 7f8a06000640 1 -- 172.21.15.103:0/2752285727 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f000cd80 con 0x7f8a0810a100 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.606+0000 7f8a06000640 1 -- 172.21.15.103:0/2752285727 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f00108e0 con 0x7f8a0810a100 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.606+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081a1190 con 0x7f8a0810a100 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.606+0000 7f8a06000640 1 -- 172.21.15.103:0/2752285727 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0018980 con 0x7f8a0810a100 2024-09-11T00:28:08.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.606+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081a7860 con 0x7f8a0810a100 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.610+0000 7f8a06000640 1 -- 172.21.15.103:0/2752285727 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f89f0010430 con 0x7f8a0810a100 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.610+0000 7f8a123ea740 1 Processor -- start 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.610+0000 7f8a123ea740 1 -- start start 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.610+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b5790 0x55936d5b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.610+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b80c0 0x55936d5ba4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a123ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55936d5286b0 con 0x55936d5b80c0 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a123ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55936d517ae0 con 0x55936d5b5790 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a123ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55936d51b8f0 con 0x55936d5bab00 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a0d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b5790 0x55936d5b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a0d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b5790 0x55936d5b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51998/0 (socket says 172.21.15.103:51998) 2024-09-11T00:28:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a0d600640 1 -- 172.21.15.103:0/802684246 learned_addr learned my addr 172.21.15.103:0/802684246 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b80c0 0x55936d5ba4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f89ffe00640 1 -- 172.21.15.103:0/802684246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b5790 msgr2=0x55936d5b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f89ffe00640 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b5790 0x55936d5b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f89ffe00640 1 -- 172.21.15.103:0/802684246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b80c0 msgr2=0x55936d5ba4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f89ffe00640 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b80c0 0x55936d5ba4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.611+0000 7f89ffe00640 1 -- 172.21.15.103:0/802684246 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936d4e4780 con 0x55936d5bab00 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.612+0000 7f8a06000640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89ec0b53e0 0x7f89ec0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.612+0000 7f89ffe00640 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5bcfc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f89f00e38d0 tx=0x7f89f00ee780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=511f32d5ea098dab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.612+0000 7f8a06000640 1 -- 172.21.15.103:0/2752285727 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f89f00e6220 con 0x7f8a0810a100 2024-09-11T00:28:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.612+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b80c0 0x55936d5ba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.612+0000 7f89fdc00640 1 -- 172.21.15.103:0/802684246 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00a8cf0 con 0x55936d5bab00 2024-09-11T00:28:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.612+0000 7f89fdc00640 1 -- 172.21.15.103:0/802684246 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f00ac1b0 con 0x55936d5bab00 2024-09-11T00:28:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.612+0000 7f89fdc00640 1 -- 172.21.15.103:0/802684246 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00abe10 con 0x55936d5bab00 2024-09-11T00:28:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.613+0000 7f8a0d600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89ec0b53e0 0x7f89ec0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.613+0000 7f8a123ea740 1 -- 172.21.15.103:0/802684246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 msgr2=0x55936d5bcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.613+0000 7f8a123ea740 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5bcfc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f89f00e38d0 tx=0x7f89f00ee780 comp rx=0 tx=0).stop 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.613+0000 7f8a0d600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89ec0b53e0 0x7f89ec0b78a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a00011650 tx=0x7f8a00009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.614+0000 7f8a123ea740 1 -- 172.21.15.103:0/802684246 shutdown_connections 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.614+0000 7f8a123ea740 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5bcfc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.614+0000 7f8a123ea740 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b80c0 0x55936d5ba4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.614+0000 7f8a123ea740 1 --2- 172.21.15.103:0/802684246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b5790 0x55936d5b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.614+0000 7f8a123ea740 1 -- 172.21.15.103:0/802684246 >> 172.21.15.103:0/802684246 conn(0x55936d5b08e0 msgr2=0x55936d5b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.614+0000 7f8a123ea740 1 -- 172.21.15.103:0/802684246 shutdown_connections 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.614+0000 7f8a123ea740 1 -- 172.21.15.103:0/802684246 wait complete. 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 -- start start 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 Processor -- start 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5bef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b80c0 0x55936d5bf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5bab00 0x55936d5c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55936d44fab0 con 0x55936d5b5790 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55936d4be570 con 0x55936d5bab00 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55936d5bdae0 con 0x55936d5b80c0 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a123ea740 10 client.0 _ll_register_callbacks cb 0x55936d4cd100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.615+0000 7f8a0d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5bab00 0x55936d5c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5bab00 0x55936d5c0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52008/0 (socket says 172.21.15.103:52008) 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 -- 172.21.15.103:0/3787418318 learned_addr learned my addr 172.21.15.103:0/3787418318 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f89ffe00640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b80c0 0x55936d5bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 -- 172.21.15.103:0/3787418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b80c0 msgr2=0x55936d5bf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b80c0 0x55936d5bf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 -- 172.21.15.103:0/3787418318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 msgr2=0x55936d5bef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5bef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 -- 172.21.15.103:0/3787418318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936d279790 con 0x55936d5bab00 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f89ffe00640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b80c0 0x55936d5bf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f8a0d600640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5bab00 0x55936d5c0180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009810 tx=0x7f8a00004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6788caafed445952 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.616+0000 7f89f6000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.617+0000 7f89f6000640 1 -- 172.21.15.103:0/3787418318 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0002d060 con 0x55936d5bab00 2024-09-11T00:28:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.617+0000 7f89f6000640 1 -- 172.21.15.103:0/3787418318 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f8a00017030 con 0x55936d5bab00 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.617+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f00ac1b0 con 0x55936d5bab00 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.617+0000 7f89f6000640 1 -- 172.21.15.103:0/3787418318 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8a00034050 con 0x55936d5bab00 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.617+0000 7f8a123ea740 10 client.25919 fetch_fsmap learned FSMap version 39 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.617+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55936d2f2620 con 0x55936d5bab00 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.617+0000 7f89f6000640 1 -- 172.21.15.103:0/3787418318 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f00ac1b0 con 0x55936d5bab00 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.618+0000 7f89f6000640 1 -- 172.21.15.103:0/3787418318 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f8a00028830 con 0x55936d5bab00 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.618+0000 7f8a123ea740 10 client.25919 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.618+0000 7f8a123ea740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.618+0000 7f8a123ea740 20 client.25919 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.618+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d5ca8d0 0x55936d5cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.619+0000 7f8a123ea740 4 client.25919 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.619+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55936d52a600 con 0x55936d5ca8d0 2024-09-11T00:28:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.619+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d5ca8d0 0x55936d5cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.620+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d5ca8d0 0x55936d5cccf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d97997a215c6ceb server_cookie=d183aa8df0595e72 in_seq=0 out_seq=0 2024-09-11T00:28:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.620+0000 7f89f6000640 10 client.25919 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.621+0000 7f89f6000640 1 -- 172.21.15.103:0/3787418318 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55936d52a600 con 0x55936d5ca8d0 2024-09-11T00:28:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.621+0000 7f89f6000640 10 client.25919 handle_command_reply: tid=0 2024-09-11T00:28:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 client.25919 shutdown 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5bab00 msgr2=0x55936d5c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5bab00 0x55936d5c0180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009810 tx=0x7f8a00004ab0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 shutdown_connections 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5bab00 0x55936d5c0180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d5ca8d0 0x55936d5cccf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b80c0 0x55936d5bf620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3787418318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5bef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f89f6000640 0 client.25919 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 >> 172.21.15.103:0/3787418318 conn(0x55936d5b08e0 msgr2=0x55936d5afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 shutdown_connections 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 1 -- 172.21.15.103:0/3787418318 wait complete. 2024-09-11T00:28:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.622+0000 7f8a123ea740 20 client.25919 trim_cache size 0 max 16384 2024-09-11T00:28:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 Processor -- start 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 -- start start 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b5790 0x55936d5ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5bab00 0x55936d5be420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b4620 con 0x55936d5bab00 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b1e60 con 0x55936d5b80c0 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.639+0000 7f8a123ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081afcb0 con 0x55936d5b5790 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f8a0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b5790 0x55936d5ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.642 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5bdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52014/0 (socket says 172.21.15.103:52014) 2024-09-11T00:28:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f8a0cc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b5790 0x55936d5ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:38906/0 (socket says 172.21.15.103:38906) 2024-09-11T00:28:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 -- 172.21.15.103:0/3450582985 learned_addr learned my addr 172.21.15.103:0/3450582985 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f8a0d600640 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5bab00 0x55936d5be420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 -- 172.21.15.103:0/3450582985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b5790 msgr2=0x55936d5ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b5790 0x55936d5ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 -- 172.21.15.103:0/3450582985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5bab00 msgr2=0x55936d5be420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5bab00 0x55936d5be420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 -- 172.21.15.103:0/3450582985 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55936d5b7ef0 con 0x55936d5b80c0 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b5790 0x55936d5ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.640+0000 7f89ffe00640 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5bdee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89f00a9910 tx=0x7f89f00e9cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c662b3c2da87d9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f89fd200640 1 -- 172.21.15.103:0/3450582985 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00abff0 con 0x55936d5b80c0 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f89fd200640 1 -- 172.21.15.103:0/3450582985 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f00ac1b0 con 0x55936d5b80c0 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f89fd200640 1 -- 172.21.15.103:0/3450582985 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00ac470 con 0x55936d5b80c0 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f8a123ea740 1 -- 172.21.15.103:0/3450582985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 msgr2=0x55936d5bdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.645 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5bdee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89f00a9910 tx=0x7f89f00e9cd0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f8a123ea740 1 -- 172.21.15.103:0/3450582985 shutdown_connections 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5bab00 0x55936d5be420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5bdee0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f8a123ea740 1 --2- 172.21.15.103:0/3450582985 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5b5790 0x55936d5ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.641+0000 7f8a123ea740 1 -- 172.21.15.103:0/3450582985 >> 172.21.15.103:0/3450582985 conn(0x55936d5b08e0 msgr2=0x55936d5baf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.642+0000 7f8a123ea740 1 -- 172.21.15.103:0/3450582985 shutdown_connections 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.642+0000 7f8a123ea740 1 -- 172.21.15.103:0/3450582985 wait complete. 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.642+0000 7f8a123ea740 1 -- start start 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.642+0000 7f8a123ea740 1 Processor -- start 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.646 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5c6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a123ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a123ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b0320 con 0x55936d5b5790 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a123ea740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b1790 con 0x55936d5b80c0 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a123ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081b09f0 con 0x55936d5bab00 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a123ea740 10 client.0 _ll_register_callbacks cb 0x7f8a00009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52020/0 (socket says 172.21.15.103:52020) 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 -- 172.21.15.103:0/159688270 learned_addr learned my addr 172.21.15.103:0/159688270 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a0d600640 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 -- 172.21.15.103:0/159688270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 msgr2=0x55936d5c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 -- 172.21.15.103:0/159688270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 msgr2=0x55936d5c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.643+0000 7f89ffe00640 1 -- 172.21.15.103:0/159688270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55936d5ba820 con 0x55936d5b80c0 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f89ffe00640 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5c6680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89f00cdfd0 tx=0x7f89f0018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab6330494c640d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f89f6000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f89f6000640 1 -- 172.21.15.103:0/159688270 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f00a8540 con 0x55936d5b80c0 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f89f6000640 1 -- 172.21.15.103:0/159688270 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f89f00b56f0 con 0x55936d5b80c0 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f00ac1b0 con 0x55936d5b80c0 2024-09-11T00:28:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f89f6000640 1 -- 172.21.15.103:0/159688270 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f89f00c3070 con 0x55936d5b80c0 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f8a123ea740 10 client.25925 fetch_fsmap learned FSMap version 39 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.644+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55936d637ef0 con 0x55936d5b80c0 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f89f6000640 1 -- 172.21.15.103:0/159688270 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89f00ac1b0 con 0x55936d5b80c0 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f89f6000640 1 -- 172.21.15.103:0/159688270 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f89f0018b20 con 0x55936d5b80c0 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f8a123ea740 10 client.25925 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f8a123ea740 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f8a123ea740 20 client.25925 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f8a123ea740 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d6382c0 0x55936d63a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f8a123ea740 4 client.25925 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55936d5260b0 con 0x55936d6382c0 2024-09-11T00:28:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.645+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d6382c0 0x55936d63a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.646+0000 7f8a0cc00640 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d6382c0 0x55936d63a6b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ed69a9cbb60160 server_cookie=658e070808a8d0ba in_seq=0 out_seq=0 2024-09-11T00:28:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.646+0000 7f89f6000640 10 client.25925 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.656+0000 7f89f6000640 1 -- 172.21.15.103:0/159688270 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55936d5260b0 con 0x55936d6382c0 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.656+0000 7f89f6000640 10 client.25925 handle_command_reply: tid=0 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.656+0000 7f8a123ea740 1 client.25925 shutdown 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 msgr2=0x55936d5c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5c6680 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89f00cdfd0 tx=0x7f89f0018420 comp rx=0 tx=0).stop 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 shutdown_connections 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55936d6382c0 0x55936d63a6b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55936d5bab00 0x55936d5c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f89f6000640 0 client.25925 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55936d5b80c0 0x55936d5c6680 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 --2- 172.21.15.103:0/159688270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55936d5b5790 0x55936d5c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 >> 172.21.15.103:0/159688270 conn(0x55936d5b08e0 msgr2=0x55936d5bdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 shutdown_connections 2024-09-11T00:28:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 1 -- 172.21.15.103:0/159688270 wait complete. 2024-09-11T00:28:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.657+0000 7f8a123ea740 20 client.25925 trim_cache size 0 max 16384 2024-09-11T00:28:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:28:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:28:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:28:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:28:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89ec0b53e0 msgr2=0x7f89ec0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89ec0b53e0 0x7f89ec0b78a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8a00011650 tx=0x7f8a00009500 comp rx=0 tx=0).stop 2024-09-11T00:28:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0810a100 msgr2=0x7f8a0819f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0810a100 0x7f8a0819f9b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f89f000e9f0 tx=0x7f89f000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 shutdown_connections 2024-09-11T00:28:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89ec0b53e0 0x7f89ec0b78a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a0810aac0 0x7f8a081a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a0810a100 0x7f8a0819f9b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 --2- 172.21.15.103:0/2752285727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08109230 0x7f8a0819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 >> 172.21.15.103:0/2752285727 conn(0x7f8a080783d0 msgr2=0x7f8a0810d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 shutdown_connections 2024-09-11T00:28:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:08.658+0000 7f8a0f600640 1 -- 172.21.15.103:0/2752285727 wait complete. 2024-09-11T00:28:08.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:08 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:08.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:08 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:08.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 -- 172.21.15.103:0/1545662291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e4075de0 msgr2=0x7f79e410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 --2- 172.21.15.103:0/1545662291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e4075de0 0x7f79e410ee70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009900 tx=0x7f79cc02a410 comp rx=0 tx=0).stop 2024-09-11T00:28:08.965 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 -- 172.21.15.103:0/1545662291 shutdown_connections 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 --2- 172.21.15.103:0/1545662291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e410f3b0 0x7f79e41117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 --2- 172.21.15.103:0/1545662291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e4075de0 0x7f79e410ee70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 --2- 172.21.15.103:0/1545662291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e40754c0 0x7f79e40758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 -- 172.21.15.103:0/1545662291 >> 172.21.15.103:0/1545662291 conn(0x7f79e40fe810 msgr2=0x7f79e4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 -- 172.21.15.103:0/1545662291 shutdown_connections 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.962+0000 7f79ea200640 1 -- 172.21.15.103:0/1545662291 wait complete. 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.963+0000 7f79ea200640 1 Processor -- start 2024-09-11T00:28:08.966 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.963+0000 7f79ea200640 1 -- start start 2024-09-11T00:28:08.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79ea200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40754c0 0x7f79e419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79ea200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e4075de0 0x7f79e419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79ea200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e410f3b0 0x7f79e41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79ea200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4114ff0 con 0x7f79e410f3b0 2024-09-11T00:28:08.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79ea200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4114e70 con 0x7f79e40754c0 2024-09-11T00:28:08.967 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79ea200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79e4115170 con 0x7f79e4075de0 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e4075de0 0x7f79e419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e4075de0 0x7f79e419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:38932/0 (socket says 172.21.15.103:38932) 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79dbe00640 1 -- 172.21.15.103:0/3522222980 learned_addr learned my addr 172.21.15.103:0/3522222980 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79e3e00640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e410f3b0 0x7f79e41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.964+0000 7f79e3400640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40754c0 0x7f79e419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79dbe00640 1 -- 172.21.15.103:0/3522222980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40754c0 msgr2=0x7f79e419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79dbe00640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40754c0 0x7f79e419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79dbe00640 1 -- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e410f3b0 msgr2=0x7f79e41a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.968 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79dbe00640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e410f3b0 0x7f79e41a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79dbe00640 1 -- 172.21.15.103:0/3522222980 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e41a1590 con 0x7f79e4075de0 2024-09-11T00:28:08.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79dbe00640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e4075de0 0x7f79e419ba20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f79cc005ab0 tx=0x7f79cc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79e0c00640 1 -- 172.21.15.103:0/3522222980 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc03f070 con 0x7f79e4075de0 2024-09-11T00:28:08.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e41a1820 con 0x7f79e4075de0 2024-09-11T00:28:08.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79e0c00640 1 -- 172.21.15.103:0/3522222980 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79cc031b60 con 0x7f79e4075de0 2024-09-11T00:28:08.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79e0c00640 1 -- 172.21.15.103:0/3522222980 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc035030 con 0x7f79e4075de0 2024-09-11T00:28:08.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.965+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e41a7f40 con 0x7f79e4075de0 2024-09-11T00:28:08.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.969+0000 7f79eccbb740 1 Processor -- start 2024-09-11T00:28:08.971 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.969+0000 7f79eccbb740 1 -- start start 2024-09-11T00:28:08.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e0c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e1ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.972 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79eccbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca94d81ca0 con 0x55ca94e18a30 2024-09-11T00:28:08.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79eccbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ca94d710d0 con 0x55ca94e0c180 2024-09-11T00:28:08.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79eccbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca94d74ee0 con 0x55ca94e0caa0 2024-09-11T00:28:08.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e0c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e0c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52056/0 (socket says 172.21.15.103:52056) 2024-09-11T00:28:08.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79e3e00640 1 -- 172.21.15.103:0/2822977440 learned_addr learned my addr 172.21.15.103:0/2822977440 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.970+0000 7f79e3400640 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79e3e00640 1 -- 172.21.15.103:0/2822977440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 msgr2=0x55ca94e184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79e3e00640 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79e3e00640 1 -- 172.21.15.103:0/2822977440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 msgr2=0x55ca94e1ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79e3e00640 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e1ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79e3e00640 1 -- 172.21.15.103:0/2822977440 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca94d23400 con 0x55ca94e0c180 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79e0c00640 1 -- 172.21.15.103:0/3522222980 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f79cc03b460 con 0x7f79e4075de0 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79e3e00640 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e0c560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009f50 tx=0x7f79dc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b933a3c6f5d8194 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79d3e00640 1 -- 172.21.15.103:0/2822977440 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc004b20 con 0x55ca94e0c180 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79d3e00640 1 -- 172.21.15.103:0/2822977440 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79dc004cc0 con 0x55ca94e0c180 2024-09-11T00:28:08.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.971+0000 7f79d3e00640 1 -- 172.21.15.103:0/2822977440 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc005020 con 0x55ca94e0c180 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.972+0000 7f79eccbb740 1 -- 172.21.15.103:0/2822977440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 msgr2=0x55ca94e0c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.972+0000 7f79eccbb740 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e0c560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009f50 tx=0x7f79dc0118d0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.973+0000 7f79eccbb740 1 -- 172.21.15.103:0/2822977440 shutdown_connections 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.973+0000 7f79eccbb740 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e1ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.973+0000 7f79eccbb740 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.973+0000 7f79eccbb740 1 --2- 172.21.15.103:0/2822977440 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e0c560 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.973+0000 7f79eccbb740 1 -- 172.21.15.103:0/2822977440 >> 172.21.15.103:0/2822977440 conn(0x55ca94e09fb0 msgr2=0x55ca94e0b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.975 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.973+0000 7f79e0c00640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79c80b51f0 0x7f79c80b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.973+0000 7f79e0c00640 1 -- 172.21.15.103:0/3522222980 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f79cc1064b0 con 0x7f79e4075de0 2024-09-11T00:28:08.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.974+0000 7f79e3400640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79c80b51f0 0x7f79c80b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.974+0000 7f79eccbb740 1 -- 172.21.15.103:0/2822977440 shutdown_connections 2024-09-11T00:28:08.976 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.974+0000 7f79eccbb740 1 -- 172.21.15.103:0/2822977440 wait complete. 2024-09-11T00:28:08.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.974+0000 7f79eccbb740 1 -- start start 2024-09-11T00:28:08.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79e3400640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79c80b51f0 0x7f79c80b76b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ca94e0ae90 tx=0x7f79d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79eccbb740 1 Processor -- start 2024-09-11T00:28:08.977 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e18410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e15900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e15ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79eccbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ca94bd27a0 con 0x55ca94e0c180 2024-09-11T00:28:08.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79eccbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55ca94d16a10 con 0x55ca94e18a30 2024-09-11T00:28:08.978 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.975+0000 7f79eccbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca94e23d70 con 0x55ca94e0caa0 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79eccbb740 10 client.0 _ll_register_callbacks cb 0x55ca94d26be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e15ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e15ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52060/0 (socket says 172.21.15.103:52060) 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 -- 172.21.15.103:0/3889678786 learned_addr learned my addr 172.21.15.103:0/3889678786 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79dbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e15900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3400640 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e18410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 -- 172.21.15.103:0/3889678786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 msgr2=0x55ca94e15900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e15900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 -- 172.21.15.103:0/3889678786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 msgr2=0x55ca94e18410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e18410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3e00640 1 -- 172.21.15.103:0/3889678786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca94adeb90 con 0x55ca94e18a30 2024-09-11T00:28:08.979 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.976+0000 7f79e3400640 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e18410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79e3e00640 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e15ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79dc012fd0 tx=0x7f79dc009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76826a6d7e2a8df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79d0c00640 1 -- 172.21.15.103:0/3889678786 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc029020 con 0x55ca94e18a30 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79d0c00640 1 -- 172.21.15.103:0/3889678786 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f79dc003d80 con 0x55ca94e18a30 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79dc004cc0 con 0x55ca94e18a30 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79d0c00640 1 -- 172.21.15.103:0/3889678786 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79dc003f80 con 0x55ca94e18a30 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79eccbb740 10 client.25937 fetch_fsmap learned FSMap version 39 2024-09-11T00:28:08.981 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.977+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca94e2f860 con 0x55ca94e18a30 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.978+0000 7f79d0c00640 1 -- 172.21.15.103:0/3889678786 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79dc004cc0 con 0x55ca94e18a30 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.978+0000 7f79d0c00640 1 -- 172.21.15.103:0/3889678786 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f79dc025470 con 0x55ca94e18a30 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.978+0000 7f79eccbb740 10 client.25937 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.978+0000 7f79eccbb740 10 client.25937 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.978+0000 7f79eccbb740 20 client.25937 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.979+0000 7f79eccbb740 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e30cc0 0x55ca94e330b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.979+0000 7f79eccbb740 4 client.25937 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.979+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca94d83bf0 con 0x55ca94e30cc0 2024-09-11T00:28:08.982 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.979+0000 7f79e3400640 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e30cc0 0x55ca94e330b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:08.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.980+0000 7f79e3400640 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e30cc0 0x55ca94e330b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91b8219958a7987 server_cookie=875cc55b99d13557 in_seq=0 out_seq=0 2024-09-11T00:28:08.983 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.980+0000 7f79d0c00640 10 client.25937 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:08.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.981+0000 7f79d0c00640 1 -- 172.21.15.103:0/3889678786 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca94d83bf0 con 0x55ca94e30cc0 2024-09-11T00:28:08.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.981+0000 7f79d0c00640 10 client.25937 handle_command_reply: tid=0 2024-09-11T00:28:08.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 client.25937 shutdown 2024-09-11T00:28:08.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 msgr2=0x55ca94e15ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:08.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e15ff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79dc012fd0 tx=0x7f79dc009be0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.984 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 shutdown_connections 2024-09-11T00:28:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e15ff0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e30cc0 0x55ca94e330b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e15900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 --2- 172.21.15.103:0/3889678786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e18410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79d0c00640 0 client.25937 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 >> 172.21.15.103:0/3889678786 conn(0x55ca94e09fb0 msgr2=0x55ca94e0fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:08.985 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 shutdown_connections 2024-09-11T00:28:08.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.982+0000 7f79eccbb740 1 -- 172.21.15.103:0/3889678786 wait complete. 2024-09-11T00:28:08.986 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:08.983+0000 7f79eccbb740 20 client.25937 trim_cache size 0 max 16384 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 Processor -- start 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 -- start start 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e178f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e12b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41b0270 con 0x55ca94e18a30 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41adab0 con 0x55ca94e0c180 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.006+0000 7f79eccbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41ab920 con 0x55ca94e0caa0 2024-09-11T00:28:09.009 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e178f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e178f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:52062/0 (socket says 172.21.15.103:52062) 2024-09-11T00:28:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 -- 172.21.15.103:0/1615425664 learned_addr learned my addr 172.21.15.103:0/1615425664 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 -- 172.21.15.103:0/1615425664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 msgr2=0x55ca94e100c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:09.010 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e100c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 -- 172.21.15.103:0/1615425664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 msgr2=0x55ca94e12b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3e00640 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e12b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e12b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3400640 1 -- 172.21.15.103:0/1615425664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca94e1b240 con 0x55ca94e0c180 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.007+0000 7f79e3e00640 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e12b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.008+0000 7f79e3400640 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e178f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ca94e336b0 tx=0x7f79d4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=719169bdbd2687a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.008+0000 7f79d3400640 1 -- 172.21.15.103:0/1615425664 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4005020 con 0x55ca94e0c180 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.008+0000 7f79d3400640 1 -- 172.21.15.103:0/1615425664 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d4003850 con 0x55ca94e0c180 2024-09-11T00:28:09.011 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.008+0000 7f79d3400640 1 -- 172.21.15.103:0/1615425664 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4004900 con 0x55ca94e0c180 2024-09-11T00:28:09.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 -- 172.21.15.103:0/1615425664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 msgr2=0x55ca94e178f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:09.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e178f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55ca94e336b0 tx=0x7f79d4017000 comp rx=0 tx=0).stop 2024-09-11T00:28:09.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 -- 172.21.15.103:0/1615425664 shutdown_connections 2024-09-11T00:28:09.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e18a30 0x55ca94e12b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.012 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 --2- 172.21.15.103:0/1615425664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e0c180 0x55ca94e178f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 -- 172.21.15.103:0/1615425664 >> 172.21.15.103:0/1615425664 conn(0x55ca94e09fb0 msgr2=0x55ca94e090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 -- 172.21.15.103:0/1615425664 shutdown_connections 2024-09-11T00:28:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.009+0000 7f79eccbb740 1 -- 172.21.15.103:0/1615425664 wait complete. 2024-09-11T00:28:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.010+0000 7f79eccbb740 1 -- start start 2024-09-11T00:28:09.013 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.010+0000 7f79eccbb740 1 Processor -- start 2024-09-11T00:28:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79eccbb740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e26f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79eccbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41abf70 con 0x55ca94e0c180 2024-09-11T00:28:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79eccbb740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41ad3e0 con 0x55ca94e18a30 2024-09-11T00:28:09.014 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79eccbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f79e41ac640 con 0x55ca94e0caa0 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79eccbb740 10 client.0 _ll_register_callbacks cb 0x7f79d40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79e3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79e3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e264d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:50142/0 (socket says 172.21.15.103:50142) 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79e3400640 1 -- 172.21.15.103:0/4283726640 learned_addr learned my addr 172.21.15.103:0/4283726640 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79dbe00640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.011+0000 7f79e3e00640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e26f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79e3400640 1 -- 172.21.15.103:0/4283726640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 msgr2=0x55ca94e26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79e3400640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79e3400640 1 -- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 msgr2=0x55ca94e26f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79e3400640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e26f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79e3400640 1 -- 172.21.15.103:0/4283726640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca94e0c8d0 con 0x55ca94e0c180 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79dbe00640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79e3400640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e264d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f79d40040b0 tx=0x7f79d4003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4a1b8e9987899a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:09.015 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.012+0000 7f79d0c00640 1 -- 172.21.15.103:0/4283726640 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4039030 con 0x55ca94e0c180 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79d0c00640 1 -- 172.21.15.103:0/4283726640 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f79d402e070 con 0x55ca94e0c180 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d4003850 con 0x55ca94e0c180 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79d0c00640 1 -- 172.21.15.103:0/4283726640 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f79d4043050 con 0x55ca94e0c180 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79eccbb740 10 client.15900 fetch_fsmap learned FSMap version 39 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ca94e9ecc0 con 0x55ca94e0c180 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79d0c00640 1 -- 172.21.15.103:0/4283726640 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d4003850 con 0x55ca94e0c180 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79d0c00640 1 -- 172.21.15.103:0/4283726640 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f79d401f700 con 0x55ca94e0c180 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79eccbb740 10 client.15900 fetch_fsmap finished waiting for FSMap version 39 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79eccbb740 10 client.15900 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:28:09.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79eccbb740 20 client.15900 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:09.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.013+0000 7f79eccbb740 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e9f530 0x55ca94ea1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:09.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.014+0000 7f79eccbb740 4 client.15900 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T00:28:09.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.014+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55ca94d7f6a0 con 0x55ca94e9f530 2024-09-11T00:28:09.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.014+0000 7f79dbe00640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e9f530 0x55ca94ea1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:09.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.014+0000 7f79dbe00640 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e9f530 0x55ca94ea1950 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60de5343c51ed34 server_cookie=ef6ff2f3ffb2de45 in_seq=0 out_seq=0 2024-09-11T00:28:09.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.014+0000 7f79d0c00640 10 client.15900 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:09.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:08 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:09.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:09 smithi103 ceph-mon[30578]: pgmap v552: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 504 KiB/s wr, 110 op/s 2024-09-11T00:28:09.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:09 smithi117 ceph-mon[35525]: pgmap v552: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 504 KiB/s wr, 110 op/s 2024-09-11T00:28:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.966+0000 7f79d0c00640 1 -- 172.21.15.103:0/4283726640 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ca94d7f6a0 con 0x55ca94e9f530 2024-09-11T00:28:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.966+0000 7f79d0c00640 10 client.15900 handle_command_reply: tid=0 2024-09-11T00:28:09.969 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.966+0000 7f79eccbb740 1 client.15900 shutdown 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 msgr2=0x55ca94e264d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e264d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f79d40040b0 tx=0x7f79d4003dd0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 shutdown_connections 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ca94e18a30 0x55ca94e26f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55ca94e9f530 0x55ca94ea1950 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca94e0caa0 0x55ca94e26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 --2- 172.21.15.103:0/4283726640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ca94e0c180 0x55ca94e264d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79d0c00640 0 client.15900 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 >> 172.21.15.103:0/4283726640 conn(0x55ca94e09fb0 msgr2=0x55ca94e0f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 shutdown_connections 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 1 -- 172.21.15.103:0/4283726640 wait complete. 2024-09-11T00:28:09.970 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.967+0000 7f79eccbb740 20 client.15900 trim_cache size 0 max 16384 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 39, 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 18, 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout: "44154a20": { 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 18, 2024-09-11T00:28:09.971 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:09.972 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:09.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:09.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:09.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:09.973 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:28:09.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79c80b51f0 msgr2=0x7f79c80b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:09.973 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79c80b51f0 0x7f79c80b76b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ca94e0ae90 tx=0x7f79d4005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e4075de0 msgr2=0x7f79e419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e4075de0 0x7f79e419ba20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f79cc005ab0 tx=0x7f79cc003c10 comp rx=0 tx=0).stop 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 shutdown_connections 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f79c80b51f0 0x7f79c80b76b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79e410f3b0 0x7f79e41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f79e4075de0 0x7f79e419ba20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 --2- 172.21.15.103:0/3522222980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e40754c0 0x7f79e419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 >> 172.21.15.103:0/3522222980 conn(0x7f79e40fe810 msgr2=0x7f79e40ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 shutdown_connections 2024-09-11T00:28:09.974 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:09.968+0000 7f79ea200640 1 -- 172.21.15.103:0/3522222980 wait complete. 2024-09-11T00:28:10.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:09 smithi183 ceph-mon[35934]: pgmap v552: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 504 KiB/s wr, 110 op/s 2024-09-11T00:28:11.235 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '44154a20', quiesce took 6.3 seconds. Will release after: 23.7 2024-09-11T00:28:11.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:11 smithi103 ceph-mon[30578]: pgmap v553: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 952 KiB/s wr, 185 op/s 2024-09-11T00:28:11.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:11 smithi117 ceph-mon[35525]: pgmap v553: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 952 KiB/s wr, 185 op/s 2024-09-11T00:28:12.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:11 smithi183 ceph-mon[35934]: pgmap v553: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 952 KiB/s wr, 185 op/s 2024-09-11T00:28:13.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:13 smithi103 ceph-mon[30578]: pgmap v554: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 952 KiB/s wr, 184 op/s 2024-09-11T00:28:13.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:13 smithi117 ceph-mon[35525]: pgmap v554: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 952 KiB/s wr, 184 op/s 2024-09-11T00:28:14.033 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:13 smithi183 ceph-mon[35934]: pgmap v554: 129 pgs: 129 active+clean; 63 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 952 KiB/s wr, 184 op/s 2024-09-11T00:28:15.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:15 smithi103 ceph-mon[30578]: pgmap v555: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-09-11T00:28:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:15 smithi117 ceph-mon[35525]: pgmap v555: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-09-11T00:28:16.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:15 smithi183 ceph-mon[35934]: pgmap v555: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 234 op/s 2024-09-11T00:28:17.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:17 smithi103 ceph-mon[30578]: pgmap v556: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 235 op/s 2024-09-11T00:28:17.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:17 smithi117 ceph-mon[35525]: pgmap v556: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 235 op/s 2024-09-11T00:28:18.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:17 smithi183 ceph-mon[35934]: pgmap v556: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.2 MiB/s wr, 235 op/s 2024-09-11T00:28:19.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:19 smithi103 ceph-mon[30578]: pgmap v557: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 230 op/s 2024-09-11T00:28:19.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:19 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:19.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:19 smithi117 ceph-mon[35525]: pgmap v557: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 230 op/s 2024-09-11T00:28:19.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:19 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:20.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:19 smithi183 ceph-mon[35934]: pgmap v557: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 230 op/s 2024-09-11T00:28:20.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:19 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 -- 172.21.15.103:0/2874272765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 msgr2=0x7f8eec107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/2874272765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec107f50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009900 tx=0x7f8ed802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 -- 172.21.15.103:0/2874272765 shutdown_connections 2024-09-11T00:28:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/2874272765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eec108690 0x7f8eec10ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/2874272765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec107f50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/2874272765 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eec06bf50 0x7f8eec1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 -- 172.21.15.103:0/2874272765 >> 172.21.15.103:0/2874272765 conn(0x7f8eec0fd150 msgr2=0x7f8eec0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 -- 172.21.15.103:0/2874272765 shutdown_connections 2024-09-11T00:28:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.785+0000 7f8ef0c00640 1 -- 172.21.15.103:0/2874272765 wait complete. 2024-09-11T00:28:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 Processor -- start 2024-09-11T00:28:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 -- start start 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eec06bf50 0x7f8eec19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eec108690 0x7f8eec19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec10e610 con 0x7f8eec105ae0 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec10e490 con 0x7f8eec06bf50 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.786+0000 7f8ef0c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec10e790 con 0x7f8eec108690 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec19b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42726/0 (socket says 172.21.15.103:42726) 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 -- 172.21.15.103:0/3402792792 learned_addr learned my addr 172.21.15.103:0/3402792792 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8eea800640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eec108690 0x7f8eec19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eec06bf50 0x7f8eec19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 -- 172.21.15.103:0/3402792792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eec108690 msgr2=0x7f8eec19c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eec108690 0x7f8eec19c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 -- 172.21.15.103:0/3402792792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eec06bf50 msgr2=0x7f8eec19afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eec06bf50 0x7f8eec19afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 -- 172.21.15.103:0/3402792792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec19caf0 con 0x7f8eec105ae0 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.787+0000 7f8ee9400640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec19b600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8031500 tx=0x7f8ed8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.788+0000 7f8ee2a00640 1 -- 172.21.15.103:0/3402792792 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed803f070 con 0x7f8eec105ae0 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.788+0000 7f8ee2a00640 1 -- 172.21.15.103:0/3402792792 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ed8031c50 con 0x7f8eec105ae0 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.788+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eec19cd80 con 0x7f8eec105ae0 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.788+0000 7f8ee2a00640 1 -- 172.21.15.103:0/3402792792 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8035030 con 0x7f8eec105ae0 2024-09-11T00:28:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.788+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eec1a34b0 con 0x7f8eec105ae0 2024-09-11T00:28:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 Processor -- start 2024-09-11T00:28:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 -- start start 2024-09-11T00:28:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf92d790 0x559eaf92fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf9300c0 0x559eaf9324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf934fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559eaf8a06b0 con 0x559eaf9300c0 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559eaf88fae0 con 0x559eaf932b00 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559eaf8938f0 con 0x559eaf92d790 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.791+0000 7f8ee2a00640 1 -- 172.21.15.103:0/3402792792 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f8ed803b460 con 0x7f8eec105ae0 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf9300c0 0x559eaf9324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf9300c0 0x559eaf9324b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42728/0 (socket says 172.21.15.103:42728) 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8eea800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf92d790 0x559eaf92fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 -- 172.21.15.103:0/2914088250 learned_addr learned my addr 172.21.15.103:0/2914088250 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9400640 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf934fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 -- 172.21.15.103:0/2914088250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf92d790 msgr2=0x559eaf92fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf92d790 0x559eaf92fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 -- 172.21.15.103:0/2914088250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 msgr2=0x559eaf934fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf934fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 -- 172.21.15.103:0/2914088250 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eaf85c780 con 0x559eaf9300c0 2024-09-11T00:28:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9400640 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf934fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8eea800640 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf92d790 0x559eaf92fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.792+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf9300c0 0x559eaf9324b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0114f0 tx=0x7f8edc0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e3d0eb3dc661d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:21.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.793+0000 7f8ee2a00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8ed00b52e0 0x7f8ed00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.793+0000 7f8ed7000640 1 -- 172.21.15.103:0/2914088250 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc01e070 con 0x559eaf9300c0 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.793+0000 7f8ed7000640 1 -- 172.21.15.103:0/2914088250 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc00fad0 con 0x559eaf9300c0 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.793+0000 7f8ed7000640 1 -- 172.21.15.103:0/2914088250 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc019b10 con 0x559eaf9300c0 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.793+0000 7f8ee2a00640 1 -- 172.21.15.103:0/3402792792 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f8ed8106970 con 0x7f8eec105ae0 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.793+0000 7f8eea800640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8ed00b52e0 0x7f8ed00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 -- 172.21.15.103:0/2914088250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf9300c0 msgr2=0x559eaf9324b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf9300c0 0x559eaf9324b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0114f0 tx=0x7f8edc0119b0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8eea800640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8ed00b52e0 0x7f8ed00b77a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8eec19bd30 tx=0x7f8ee4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:21.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 -- 172.21.15.103:0/2914088250 shutdown_connections 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf934fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf9300c0 0x559eaf9324b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/2914088250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf92d790 0x559eaf92fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 -- 172.21.15.103:0/2914088250 >> 172.21.15.103:0/2914088250 conn(0x559eaf9288e0 msgr2=0x559eaf929670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 -- 172.21.15.103:0/2914088250 shutdown_connections 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 -- 172.21.15.103:0/2914088250 wait complete. 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 -- start start 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.794+0000 7f8ef3c75740 1 Processor -- start 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf936f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf9375c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf938120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559eaf7c7ab0 con 0x559eaf92d790 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x559eaf836570 con 0x559eaf932b00 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559eaf935ae0 con 0x559eaf9300c0 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ef3c75740 10 client.0 _ll_register_callbacks cb 0x559eaf845100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf936f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf936f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42740/0 (socket says 172.21.15.103:42740) 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3772641894 learned_addr learned my addr 172.21.15.103:0/3772641894 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8eea800640 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf938120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.795+0000 7f8ee9400640 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf9375c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3772641894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 msgr2=0x559eaf9375c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf9375c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 msgr2=0x559eaf938120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf938120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3772641894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eaf5f1790 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf936f20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0114c0 tx=0x7f8edc00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a92078a2851e27 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ecb400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ecb400640 1 -- 172.21.15.103:0/3772641894 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc01e040 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8edc00fad0 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.796+0000 7f8ecb400640 1 -- 172.21.15.103:0/3772641894 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f8edc004a40 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ecb400640 1 -- 172.21.15.103:0/3772641894 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8edc032050 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 10 client.15912 fetch_fsmap learned FSMap version 40 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559eaf66a620 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ecb400640 1 -- 172.21.15.103:0/3772641894 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc00fad0 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ecb400640 1 -- 172.21.15.103:0/3772641894 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f8edc021030 con 0x559eaf92d790 2024-09-11T00:28:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 10 client.15912 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 20 client.15912 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf942570 0x559eaf9449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 4 client.15912 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.797+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eaf8a2600 con 0x559eaf942570 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.798+0000 7f8ee9400640 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf942570 0x559eaf9449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.798+0000 7f8ee9400640 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf942570 0x559eaf9449d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80571c202c94d070 server_cookie=626aa318844a5bc7 in_seq=0 out_seq=0 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.798+0000 7f8ecb400640 10 client.15912 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ecb400640 1 -- 172.21.15.103:0/3772641894 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559eaf8a2600 con 0x559eaf942570 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ecb400640 10 client.15912 handle_command_reply: tid=0 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 client.15912 shutdown 2024-09-11T00:28:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 msgr2=0x559eaf936f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf936f20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0114c0 tx=0x7f8edc00f850 comp rx=0 tx=0).stop 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 shutdown_connections 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf932b00 0x559eaf938120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf942570 0x559eaf9449d0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf9375c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3772641894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf936f20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ecb400640 0 client.15912 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 >> 172.21.15.103:0/3772641894 conn(0x559eaf9288e0 msgr2=0x559eaf927a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.799+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 shutdown_connections 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.800+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3772641894 wait complete. 2024-09-11T00:28:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.800+0000 7f8ef3c75740 20 client.15912 trim_cache size 0 max 16384 2024-09-11T00:28:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 Processor -- start 2024-09-11T00:28:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 -- start start 2024-09-11T00:28:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf932ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf9300c0 0x559eaf933420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf932b00 0x559eaf92a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.818 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1b0220 con 0x559eaf92d790 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1ada90 con 0x559eaf9300c0 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1ab8e0 con 0x559eaf932b00 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf932ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf932ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42752/0 (socket says 172.21.15.103:42752) 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3383434427 learned_addr learned my addr 172.21.15.103:0/3383434427 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8eea800640 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf932b00 0x559eaf92a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.815+0000 7f8ee9400640 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf9300c0 0x559eaf933420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3383434427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf932b00 msgr2=0x559eaf92a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf932b00 0x559eaf92a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3383434427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf9300c0 msgr2=0x559eaf933420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf9300c0 0x559eaf933420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ee9e00640 1 -- 172.21.15.103:0/3383434427 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eaf92fef0 con 0x559eaf92d790 2024-09-11T00:28:21.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf932ee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005d70 tx=0x7f8edc00f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fe5e2c13b80fcd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ed6600640 1 -- 172.21.15.103:0/3383434427 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc01e040 con 0x559eaf92d790 2024-09-11T00:28:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ed6600640 1 -- 172.21.15.103:0/3383434427 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc029070 con 0x559eaf92d790 2024-09-11T00:28:21.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.816+0000 7f8ed6600640 1 -- 172.21.15.103:0/3383434427 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc02e450 con 0x559eaf92d790 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3383434427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 msgr2=0x559eaf932ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf932ee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8edc005d70 tx=0x7f8edc00f940 comp rx=0 tx=0).stop 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3383434427 shutdown_connections 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf932b00 0x559eaf92a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf9300c0 0x559eaf933420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/3383434427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf92d790 0x559eaf932ee0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3383434427 >> 172.21.15.103:0/3383434427 conn(0x559eaf9288e0 msgr2=0x559eaf929330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3383434427 shutdown_connections 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 -- 172.21.15.103:0/3383434427 wait complete. 2024-09-11T00:28:21.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 -- start start 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.817+0000 7f8ef3c75740 1 Processor -- start 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf92d790 0x559eaf93dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf93e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ef3c75740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf932b00 0x559eaf93e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1abf50 con 0x559eaf932b00 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1ad3c0 con 0x559eaf92d790 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ef3c75740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec1ac620 con 0x559eaf9300c0 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ef3c75740 10 client.0 _ll_register_callbacks cb 0x7f8edc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf932b00 0x559eaf93e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf932b00 0x559eaf93e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42758/0 (socket says 172.21.15.103:42758) 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ee9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf92d790 0x559eaf93dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 -- 172.21.15.103:0/1633436452 learned_addr learned my addr 172.21.15.103:0/1633436452 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8ee9400640 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf93e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.825 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 -- 172.21.15.103:0/1633436452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 msgr2=0x559eaf93e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf93e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 -- 172.21.15.103:0/1633436452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf92d790 msgr2=0x559eaf93dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf92d790 0x559eaf93dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.818+0000 7f8eea800640 1 -- 172.21.15.103:0/1633436452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eaf93c9b0 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8eea800640 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf932b00 0x559eaf93e670 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4012cb0 tx=0x7f8ee4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956ab431cb101143 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ecb400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ecb400640 1 -- 172.21.15.103:0/1633436452 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee402e030 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ecb400640 1 -- 172.21.15.103:0/1633436452 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f8ee4018030 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8edc029070 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ecb400640 1 -- 172.21.15.103:0/1633436452 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8ee4038050 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ef3c75740 10 client.15918 fetch_fsmap learned FSMap version 40 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559eaf9afb10 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ecb400640 1 -- 172.21.15.103:0/1633436452 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc029070 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.819+0000 7f8ecb400640 1 -- 172.21.15.103:0/1633436452 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 40) ==== 8366+0+0 (secure 0 0 0) 0x7f8ee4004bc0 con 0x559eaf932b00 2024-09-11T00:28:21.826 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.820+0000 7f8ef3c75740 10 client.15918 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.820+0000 7f8ef3c75740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.820+0000 7f8ef3c75740 20 client.15918 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.820+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf9b0570 0x559eaf9b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.820+0000 7f8ef3c75740 4 client.15918 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.820+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559eaf89e0b0 con 0x559eaf9b0570 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.820+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf9b0570 0x559eaf9b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.821+0000 7f8ee9e00640 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf9b0570 0x559eaf9b2990 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79af05523c528669 server_cookie=94a4998e322ab2b5 in_seq=0 out_seq=0 2024-09-11T00:28:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.821+0000 7f8ecb400640 10 client.15918 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ecb400640 1 -- 172.21.15.103:0/1633436452 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559eaf89e0b0 con 0x559eaf9b0570 2024-09-11T00:28:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ecb400640 10 client.15918 handle_command_reply: tid=0 2024-09-11T00:28:21.834 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 client.15918 shutdown 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf932b00 msgr2=0x559eaf93e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf932b00 0x559eaf93e670 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4012cb0 tx=0x7f8ee4004730 comp rx=0 tx=0).stop 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 shutdown_connections 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559eaf932b00 0x559eaf93e670 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x559eaf9b0570 0x559eaf9b2990 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559eaf9300c0 0x559eaf93e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 --2- 172.21.15.103:0/1633436452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x559eaf92d790 0x559eaf93dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 >> 172.21.15.103:0/1633436452 conn(0x559eaf9288e0 msgr2=0x559eaf929140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ecb400640 0 client.15918 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 shutdown_connections 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 1 -- 172.21.15.103:0/1633436452 wait complete. 2024-09-11T00:28:21.835 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.832+0000 7f8ef3c75740 20 client.15918 trim_cache size 0 max 16384 2024-09-11T00:28:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:28:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:28:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:28:21.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:28:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8ed00b52e0 msgr2=0x7f8ed00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8ed00b52e0 0x7f8ed00b77a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8eec19bd30 tx=0x7f8ee4005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 msgr2=0x7f8eec19b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec19b600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8031500 tx=0x7f8ed8003c10 comp rx=0 tx=0).stop 2024-09-11T00:28:21.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 shutdown_connections 2024-09-11T00:28:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f8ed00b52e0 0x7f8ed00b77a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8eec108690 0x7f8eec19c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8eec105ae0 0x7f8eec19b600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 --2- 172.21.15.103:0/3402792792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8eec06bf50 0x7f8eec19afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 >> 172.21.15.103:0/3402792792 conn(0x7f8eec0fd150 msgr2=0x7f8eec0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 shutdown_connections 2024-09-11T00:28:21.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:21.833+0000 7f8ef0c00640 1 -- 172.21.15.103:0/3402792792 wait complete. 2024-09-11T00:28:21.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:21 smithi103 ceph-mon[30578]: pgmap v558: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 697 KiB/s wr, 135 op/s 2024-09-11T00:28:21.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:21 smithi117 ceph-mon[35525]: pgmap v558: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 697 KiB/s wr, 135 op/s 2024-09-11T00:28:22.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:21 smithi183 ceph-mon[35934]: pgmap v558: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 697 KiB/s wr, 135 op/s 2024-09-11T00:28:23.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:23 smithi103 ceph-mon[30578]: pgmap v559: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 246 KiB/s wr, 58 op/s 2024-09-11T00:28:23.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:23 smithi117 ceph-mon[35525]: pgmap v559: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 246 KiB/s wr, 58 op/s 2024-09-11T00:28:24.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:23 smithi183 ceph-mon[35934]: pgmap v559: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 246 KiB/s wr, 58 op/s 2024-09-11T00:28:25.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:25 smithi103 ceph-mon[30578]: pgmap v560: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 248 KiB/s wr, 60 op/s 2024-09-11T00:28:25.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:25 smithi117 ceph-mon[35525]: pgmap v560: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 248 KiB/s wr, 60 op/s 2024-09-11T00:28:26.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:25 smithi183 ceph-mon[35934]: pgmap v560: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 248 KiB/s wr, 60 op/s 2024-09-11T00:28:27.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:27 smithi103 ceph-mon[30578]: pgmap v561: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-09-11T00:28:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:27 smithi117 ceph-mon[35525]: pgmap v561: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-09-11T00:28:28.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:27 smithi183 ceph-mon[35934]: pgmap v561: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 20 KiB/s wr, 11 op/s 2024-09-11T00:28:28.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:28 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:28.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:28 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:29.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:28 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:28:29.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:29 smithi103 ceph-mon[30578]: pgmap v562: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-11T00:28:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:29 smithi117 ceph-mon[35525]: pgmap v562: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-11T00:28:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:29 smithi183 ceph-mon[35934]: pgmap v562: 129 pgs: 129 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-11T00:28:31.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:31 smithi103 ceph-mon[30578]: pgmap v563: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-11T00:28:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:31 smithi117 ceph-mon[35525]: pgmap v563: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-11T00:28:32.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:31 smithi183 ceph-mon[35934]: pgmap v563: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-09-11T00:28:33.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:33 smithi103 ceph-mon[30578]: pgmap v564: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 28 KiB/s wr, 8 op/s 2024-09-11T00:28:33.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:33 smithi117 ceph-mon[35525]: pgmap v564: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 28 KiB/s wr, 8 op/s 2024-09-11T00:28:34.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:33 smithi183 ceph-mon[35934]: pgmap v564: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 28 KiB/s wr, 8 op/s 2024-09-11T00:28:34.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:28:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 -- 172.21.15.103:0/1483061344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c102990 msgr2=0x7fb68c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 --2- 172.21.15.103:0/1483061344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c102990 0x7fb68c102d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009c10 tx=0x7fb67c02a6f0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 -- 172.21.15.103:0/1483061344 shutdown_connections 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 --2- 172.21.15.103:0/1483061344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb68c1044a0 0x7fb68c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 --2- 172.21.15.103:0/1483061344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c103ae0 0x7fb68c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 --2- 172.21.15.103:0/1483061344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c102990 0x7fb68c102d90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 -- 172.21.15.103:0/1483061344 >> 172.21.15.103:0/1483061344 conn(0x7fb68c0fe250 msgr2=0x7fb68c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 -- 172.21.15.103:0/1483061344 shutdown_connections 2024-09-11T00:28:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.960+0000 7fb692000640 1 -- 172.21.15.103:0/1483061344 wait complete. 2024-09-11T00:28:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 Processor -- start 2024-09-11T00:28:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 -- start start 2024-09-11T00:28:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c102990 0x7fb68c19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb68c103ae0 0x7fb68c19b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c1044a0 0x7fb68c19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c10e600 con 0x7fb68c102990 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c10e480 con 0x7fb68c103ae0 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.961+0000 7fb692000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c10e780 con 0x7fb68c1044a0 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c102990 0x7fb68c19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c102990 0x7fb68c19afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39642/0 (socket says 172.21.15.103:39642) 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb682a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb68c103ae0 0x7fb68c19b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 -- 172.21.15.103:0/3644102462 learned_addr learned my addr 172.21.15.103:0/3644102462 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68be00640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c1044a0 0x7fb68c19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 -- 172.21.15.103:0/3644102462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c1044a0 msgr2=0x7fb68c19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c1044a0 0x7fb68c19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 -- 172.21.15.103:0/3644102462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb68c103ae0 msgr2=0x7fb68c19b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb68c103ae0 0x7fb68c19b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb68b400640 1 -- 172.21.15.103:0/3644102462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68c19cb00 con 0x7fb68c102990 2024-09-11T00:28:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.962+0000 7fb682a00640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb68c103ae0 0x7fb68c19b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.963+0000 7fb68b400640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c102990 0x7fb68c19afc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000f80 tx=0x7fb67c030d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.963+0000 7fb688c00640 1 -- 172.21.15.103:0/3644102462 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c041020 con 0x7fb68c102990 2024-09-11T00:28:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.963+0000 7fb688c00640 1 -- 172.21.15.103:0/3644102462 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb67c0318f0 con 0x7fb68c102990 2024-09-11T00:28:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.963+0000 7fb688c00640 1 -- 172.21.15.103:0/3644102462 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c036040 con 0x7fb68c102990 2024-09-11T00:28:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.963+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68c19cd90 con 0x7fb68c102990 2024-09-11T00:28:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.963+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb68c1a34c0 con 0x7fb68c102990 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.966+0000 7fb694de8740 1 Processor -- start 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb694de8740 1 -- start start 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb694de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e976b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb694de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 0x558d7e9794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb694de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb694de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d7e8e76b0 con 0x558d7e9770c0 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb694de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d7e8d6ae0 con 0x558d7e979b00 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb694de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558d7e8da8f0 con 0x558d7e974790 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb68be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e976b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb68be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e976b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:41686/0 (socket says 172.21.15.103:41686) 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb68be00640 1 -- 172.21.15.103:0/831772911 learned_addr learned my addr 172.21.15.103:0/831772911 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb682a00640 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.967+0000 7fb68be00640 1 -- 172.21.15.103:0/831772911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 msgr2=0x558d7e97bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb68be00640 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb68be00640 1 -- 172.21.15.103:0/831772911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 msgr2=0x558d7e9794b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb68be00640 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 0x558d7e9794b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb68be00640 1 -- 172.21.15.103:0/831772911 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d7e8a3780 con 0x558d7e974790 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb688c00640 1 -- 172.21.15.103:0/3644102462 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fb67c004680 con 0x7fb68c102990 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb68be00640 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e976b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb684009c20 tx=0x7fb684011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=693b21e52d71af61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb67be00640 1 -- 172.21.15.103:0/831772911 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684005220 con 0x558d7e974790 2024-09-11T00:28:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb67be00640 1 -- 172.21.15.103:0/831772911 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb684004c50 con 0x558d7e974790 2024-09-11T00:28:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.968+0000 7fb67be00640 1 -- 172.21.15.103:0/831772911 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684019690 con 0x558d7e974790 2024-09-11T00:28:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 -- 172.21.15.103:0/831772911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 msgr2=0x558d7e976b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e976b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb684009c20 tx=0x7fb684011810 comp rx=0 tx=0).stop 2024-09-11T00:28:34.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 -- 172.21.15.103:0/831772911 shutdown_connections 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 0x558d7e9794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 --2- 172.21.15.103:0/831772911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e976b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 -- 172.21.15.103:0/831772911 >> 172.21.15.103:0/831772911 conn(0x558d7e96f8e0 msgr2=0x558d7e970670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb688c00640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb6700b52e0 0x7fb6700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.969+0000 7fb694de8740 1 -- 172.21.15.103:0/831772911 shutdown_connections 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.970+0000 7fb694de8740 1 -- 172.21.15.103:0/831772911 wait complete. 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.970+0000 7fb688c00640 1 -- 172.21.15.103:0/3644102462 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb67c106fb0 con 0x7fb68c102990 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.970+0000 7fb682a00640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb6700b52e0 0x7fb6700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.970+0000 7fb694de8740 1 -- start start 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.970+0000 7fb694de8740 1 Processor -- start 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.970+0000 7fb682a00640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb6700b52e0 0x7fb6700b77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb674009530 tx=0x7fb674009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb694de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e97df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb694de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 0x558d7e97e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb694de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb694de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d7e80eab0 con 0x558d7e9770c0 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb694de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558d7e87d570 con 0x558d7e979b00 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb694de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558d7e97cae0 con 0x558d7e974790 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb694de8740 10 client.0 _ll_register_callbacks cb 0x558d7e88c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb682a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 0x558d7e97e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb68b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e97df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb68b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e97df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:41692/0 (socket says 172.21.15.103:41692) 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb68be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.971+0000 7fb68b400640 1 -- 172.21.15.103:0/1784089014 learned_addr learned my addr 172.21.15.103:0/1784089014 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb68b400640 1 -- 172.21.15.103:0/1784089014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 msgr2=0x558d7e97f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb68b400640 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb68b400640 1 -- 172.21.15.103:0/1784089014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 msgr2=0x558d7e97e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb68b400640 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 0x558d7e97e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb68b400640 1 -- 172.21.15.103:0/1784089014 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d7e638790 con 0x558d7e974790 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb68b400640 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e97df00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb67c02ac10 tx=0x7fb67c0c9280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc17cfc9ce39e0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb678c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb678c00640 1 -- 172.21.15.103:0/1784089014 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0ccb50 con 0x558d7e974790 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb684004c50 con 0x558d7e974790 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb678c00640 1 -- 172.21.15.103:0/1784089014 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb67c104de0 con 0x558d7e974790 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb678c00640 1 -- 172.21.15.103:0/1784089014 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb67c0ccef0 con 0x558d7e974790 2024-09-11T00:28:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb694de8740 10 client.25999 fetch_fsmap learned FSMap version 41 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.972+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558d7e6b1620 con 0x558d7e974790 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb678c00640 1 -- 172.21.15.103:0/1784089014 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb684004c50 con 0x558d7e974790 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb678c00640 1 -- 172.21.15.103:0/1784089014 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fb67c0cc640 con 0x558d7e974790 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb694de8740 10 client.25999 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb694de8740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb694de8740 20 client.25999 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb694de8740 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e989780 0x558d7e98bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb694de8740 4 client.25999 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d7e8e9600 con 0x558d7e989780 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.973+0000 7fb682a00640 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e989780 0x558d7e98bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.974+0000 7fb682a00640 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e989780 0x558d7e98bbe0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f177e71c288a0b server_cookie=4e6303fa53a9acea in_seq=0 out_seq=0 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.974+0000 7fb678c00640 10 client.25999 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:34.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb678c00640 1 -- 172.21.15.103:0/1784089014 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558d7e8e9600 con 0x558d7e989780 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb678c00640 10 client.25999 handle_command_reply: tid=0 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 client.25999 shutdown 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 msgr2=0x558d7e97df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e97df00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb67c02ac10 tx=0x7fb67c0c9280 comp rx=0 tx=0).stop 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 shutdown_connections 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e979b00 0x558d7e97f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e9770c0 0x558d7e97e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e989780 0x558d7e98bbe0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 --2- 172.21.15.103:0/1784089014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e97df00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 >> 172.21.15.103:0/1784089014 conn(0x558d7e96f8e0 msgr2=0x558d7e96ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb678c00640 0 client.25999 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 shutdown_connections 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.975+0000 7fb694de8740 1 -- 172.21.15.103:0/1784089014 wait complete. 2024-09-11T00:28:34.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.976+0000 7fb694de8740 20 client.25999 trim_cache size 0 max 16384 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 Processor -- start 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 -- start start 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e974790 0x558d7e97dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e9770c0 0x558d7e9861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e979ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c1b0170 con 0x558d7e979b00 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c1ad9e0 con 0x558d7e974790 2024-09-11T00:28:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb694de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c1ab830 con 0x558d7e9770c0 2024-09-11T00:28:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.991+0000 7fb682a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e9770c0 0x558d7e9861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb68b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e974790 0x558d7e97dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e9770c0 0x558d7e9861f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:41702/0 (socket says 172.21.15.103:41702) 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 -- 172.21.15.103:0/3729918187 learned_addr learned my addr 172.21.15.103:0/3729918187 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb68be00640 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e979ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 -- 172.21.15.103:0/3729918187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e974790 msgr2=0x558d7e97dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e974790 0x558d7e97dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 -- 172.21.15.103:0/3729918187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 msgr2=0x558d7e979ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e979ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 -- 172.21.15.103:0/3729918187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d7e976ef0 con 0x558d7e9770c0 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb682a00640 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e9770c0 0x558d7e9861f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x558d7e98c460 tx=0x7fb674011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cf0330fa791067f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb67b400640 1 -- 172.21.15.103:0/3729918187 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674004020 con 0x558d7e9770c0 2024-09-11T00:28:34.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.992+0000 7fb67b400640 1 -- 172.21.15.103:0/3729918187 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6740041c0 con 0x558d7e9770c0 2024-09-11T00:28:34.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb67b400640 1 -- 172.21.15.103:0/3729918187 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67401c830 con 0x558d7e9770c0 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 -- 172.21.15.103:0/3729918187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e9770c0 msgr2=0x558d7e9861f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e9770c0 0x558d7e9861f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x558d7e98c460 tx=0x7fb674011f90 comp rx=0 tx=0).stop 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 -- 172.21.15.103:0/3729918187 shutdown_connections 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e979ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e9770c0 0x558d7e9861f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 --2- 172.21.15.103:0/3729918187 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e974790 0x558d7e97dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 -- 172.21.15.103:0/3729918187 >> 172.21.15.103:0/3729918187 conn(0x558d7e96f8e0 msgr2=0x558d7e970380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.993+0000 7fb694de8740 1 -- 172.21.15.103:0/3729918187 shutdown_connections 2024-09-11T00:28:34.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 -- 172.21.15.103:0/3729918187 wait complete. 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 -- start start 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 Processor -- start 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e985f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e9770c0 0x558d7e984c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e985270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c1abea0 con 0x558d7e979b00 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c1ad310 con 0x558d7e9770c0 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb68c1ac570 con 0x558d7e974790 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.994+0000 7fb694de8740 10 client.0 _ll_register_callbacks cb 0x7fb67c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb68be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e985270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e9770c0 0x558d7e984c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e9770c0 0x558d7e984c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45404/0 (socket says 172.21.15.103:45404) 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 -- 172.21.15.103:0/2145892140 learned_addr learned my addr 172.21.15.103:0/2145892140 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb68be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e985270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39664/0 (socket says 172.21.15.103:39664) 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb68b400640 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e985f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 -- 172.21.15.103:0/2145892140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 msgr2=0x558d7e985f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e985f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 -- 172.21.15.103:0/2145892140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 msgr2=0x558d7e985270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e985270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.995+0000 7fb682a00640 1 -- 172.21.15.103:0/2145892140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d7e985aa0 con 0x558d7e9770c0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb682a00640 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e9770c0 0x558d7e984c90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb674004450 tx=0x7fb674004480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94667696d44ff06a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb678c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb678c00640 1 -- 172.21.15.103:0/2145892140 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67403c060 con 0x558d7e9770c0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6740041c0 con 0x558d7e9770c0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb678c00640 1 -- 172.21.15.103:0/2145892140 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fb674005040 con 0x558d7e9770c0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb678c00640 1 -- 172.21.15.103:0/2145892140 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb674043050 con 0x558d7e9770c0 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb694de8740 10 client.25979 fetch_fsmap learned FSMap version 41 2024-09-11T00:28:34.999 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558d7e9f6860 con 0x558d7e9770c0 2024-09-11T00:28:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.996+0000 7fb678c00640 1 -- 172.21.15.103:0/2145892140 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6740041c0 con 0x558d7e9770c0 2024-09-11T00:28:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb678c00640 1 -- 172.21.15.103:0/2145892140 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7fb67402e070 con 0x558d7e9770c0 2024-09-11T00:28:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb694de8740 10 client.25979 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:28:35.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb694de8740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb694de8740 20 client.25979 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb694de8740 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e9f7190 0x558d7e9f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb694de8740 4 client.25979 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d7e8e50b0 con 0x558d7e9f7190 2024-09-11T00:28:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb68b400640 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e9f7190 0x558d7e9f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb68b400640 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e9f7190 0x558d7e9f95b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b8f47862931ae6 server_cookie=bd56456a60785fb0 in_seq=0 out_seq=0 2024-09-11T00:28:35.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:34.997+0000 7fb678c00640 10 client.25979 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.007+0000 7fb678c00640 1 -- 172.21.15.103:0/2145892140 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d7e8e50b0 con 0x558d7e9f7190 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.007+0000 7fb678c00640 10 client.25979 handle_command_reply: tid=0 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.007+0000 7fb694de8740 1 client.25979 shutdown 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e9770c0 msgr2=0x558d7e984c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e9770c0 0x558d7e984c90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb674004450 tx=0x7fb674004480 comp rx=0 tx=0).stop 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 shutdown_connections 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d7e979b00 0x558d7e985270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558d7e9770c0 0x558d7e984c90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558d7e9f7190 0x558d7e9f95b0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 --2- 172.21.15.103:0/2145892140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558d7e974790 0x558d7e985f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 >> 172.21.15.103:0/2145892140 conn(0x558d7e96f8e0 msgr2=0x558d7e973850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb678c00640 0 client.25979 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 shutdown_connections 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 1 -- 172.21.15.103:0/2145892140 wait complete. 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.008+0000 7fb694de8740 20 client.25979 trim_cache size 0 max 16384 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:28:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:28:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:28:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb6700b52e0 msgr2=0x7fb6700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb6700b52e0 0x7fb6700b77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb674009530 tx=0x7fb674009260 comp rx=0 tx=0).stop 2024-09-11T00:28:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c102990 msgr2=0x7fb68c19afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c102990 0x7fb68c19afc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb67c000f80 tx=0x7fb67c030d10 comp rx=0 tx=0).stop 2024-09-11T00:28:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 shutdown_connections 2024-09-11T00:28:35.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fb6700b52e0 0x7fb6700b77a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb68c1044a0 0x7fb68c19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb68c103ae0 0x7fb68c19b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 --2- 172.21.15.103:0/3644102462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb68c102990 0x7fb68c19afc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 >> 172.21.15.103:0/3644102462 conn(0x7fb68c0fe250 msgr2=0x7fb68c109270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 shutdown_connections 2024-09-11T00:28:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:35.009+0000 7fb692000640 1 -- 172.21.15.103:0/3644102462 wait complete. 2024-09-11T00:28:35.327 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:35.877 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:35 smithi103 ceph-mon[30578]: pgmap v565: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-09-11T00:28:35.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:35 smithi117 ceph-mon[35525]: pgmap v565: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-09-11T00:28:35.993 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 -- 172.21.15.103:0/538705769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844113b20 msgr2=0x7fd844115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 --2- 172.21.15.103:0/538705769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844113b20 0x7fd844115f10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd840009ca0 tx=0x7fd84002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 -- 172.21.15.103:0/538705769 shutdown_connections 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 --2- 172.21.15.103:0/538705769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844113b20 0x7fd844115f10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 --2- 172.21.15.103:0/538705769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd844075e60 0x7fd8441135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 --2- 172.21.15.103:0/538705769 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075540 0x7fd844075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 -- 172.21.15.103:0/538705769 >> 172.21.15.103:0/538705769 conn(0x7fd8440fe830 msgr2=0x7fd844100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 -- 172.21.15.103:0/538705769 shutdown_connections 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.987+0000 7fd84c800640 1 -- 172.21.15.103:0/538705769 wait complete. 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 Processor -- start 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 -- start start 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844075540 0x7fd84419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075e60 0x7fd84419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:35.994 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd844113b20 0x7fd8441a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441193e0 con 0x7fd844075540 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd844119260 con 0x7fd844075e60 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.988+0000 7fd84c800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd844119560 con 0x7fd844113b20 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075e60 0x7fd84419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844075540 0x7fd84419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075e60 0x7fd84419fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45422/0 (socket says 172.21.15.103:45422) 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 -- 172.21.15.103:0/3011103010 learned_addr learned my addr 172.21.15.103:0/3011103010 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd84a800640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd844113b20 0x7fd8441a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 -- 172.21.15.103:0/3011103010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd844113b20 msgr2=0x7fd8441a5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd844113b20 0x7fd8441a5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 -- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844075540 msgr2=0x7fd84419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844075540 0x7fd84419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 -- 172.21.15.103:0/3011103010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8441a5940 con 0x7fd844075e60 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849e00640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844075540 0x7fd84419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:35.995 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.989+0000 7fd849400640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075e60 0x7fd84419fdd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9f0 tx=0x7fd83400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:35.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.990+0000 7fd83ea00640 1 -- 172.21.15.103:0/3011103010 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83400cd80 con 0x7fd844075e60 2024-09-11T00:28:35.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.990+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8441a5c30 con 0x7fd844075e60 2024-09-11T00:28:35.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.990+0000 7fd83ea00640 1 -- 172.21.15.103:0/3011103010 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8340054d0 con 0x7fd844075e60 2024-09-11T00:28:35.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.990+0000 7fd83ea00640 1 -- 172.21.15.103:0/3011103010 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834010690 con 0x7fd844075e60 2024-09-11T00:28:35.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.990+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441ac2f0 con 0x7fd844075e60 2024-09-11T00:28:35.996 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.991+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd844075920 con 0x7fd844075e60 2024-09-11T00:28:35.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.993+0000 7fd83ea00640 1 -- 172.21.15.103:0/3011103010 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd834004e30 con 0x7fd844075e60 2024-09-11T00:28:35.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.995+0000 7fd83ea00640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd82c0b52e0 0x7fd82c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:35.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.996+0000 7fd83ea00640 1 -- 172.21.15.103:0/3011103010 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fd8340e65a0 con 0x7fd844075e60 2024-09-11T00:28:35.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.996+0000 7fd849e00640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd82c0b52e0 0x7fd82c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:35.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.996+0000 7fd83ea00640 1 -- 172.21.15.103:0/3011103010 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd8340ad070 con 0x7fd844075e60 2024-09-11T00:28:35.997 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:35.996+0000 7fd849e00640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd82c0b52e0 0x7fd82c0b77a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd838009fc0 tx=0x7fd838005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:36.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:35 smithi183 ceph-mon[35934]: pgmap v565: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-09-11T00:28:36.218 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.217+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8441ac680 con 0x7fd844075e60 2024-09-11T00:28:36.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.218+0000 7fd83ea00640 1 -- 172.21.15.103:0/3011103010 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11936 (secure 0 0 0) 0x7fd8340ac970 con 0x7fd844075e60 2024-09-11T00:28:36.220 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:28:36.220 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":41,"btime":"2024-09-11T00:28:27:626120+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:28:27.346017+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[1,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25858":{"gid":25858,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2940154743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2940154743},{"type":"v1","addr":"172.21.15.183:6835","nonce":2940154743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:28:36.220 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 41 2024-09-11T00:28:36.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd82c0b52e0 msgr2=0x7fd82c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:36.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd82c0b52e0 0x7fd82c0b77a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd838009fc0 tx=0x7fd838005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:36.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075e60 msgr2=0x7fd84419fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:36.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075e60 0x7fd84419fdd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9f0 tx=0x7fd83400edf0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 shutdown_connections 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd82c0b52e0 0x7fd82c0b77a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd844113b20 0x7fd8441a5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd844075e60 0x7fd84419fdd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 --2- 172.21.15.103:0/3011103010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd844075540 0x7fd84419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 >> 172.21.15.103:0/3011103010 conn(0x7fd8440fe830 msgr2=0x7fd8440ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 shutdown_connections 2024-09-11T00:28:36.223 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:36.221+0000 7fd84c800640 1 -- 172.21.15.103:0/3011103010 wait complete. 2024-09-11T00:28:36.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:36 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3011103010' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:36.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:36 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3011103010' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:37.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:36 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3011103010' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:28:37.141 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --set-id 44154a20 --release --await' 2024-09-11T00:28:37.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --set-id 44154a20 --release --await 2024-09-11T00:28:37.556 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:28:37.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:37 smithi103 ceph-mon[30578]: pgmap v566: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 27 KiB/s wr, 6 op/s 2024-09-11T00:28:37.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:37 smithi117 ceph-mon[35525]: pgmap v566: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 27 KiB/s wr, 6 op/s 2024-09-11T00:28:38.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:37 smithi183 ceph-mon[35934]: pgmap v566: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 27 KiB/s wr, 6 op/s 2024-09-11T00:28:38.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.112+0000 7f3d66000640 1 -- 172.21.15.103:0/1205033955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d601087d0 msgr2=0x7f3d60108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.112+0000 7f3d66000640 1 --2- 172.21.15.103:0/1205033955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d601087d0 0x7f3d60108c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009900 tx=0x7f3d4802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.113+0000 7f3d66000640 1 -- 172.21.15.103:0/1205033955 shutdown_connections 2024-09-11T00:28:38.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.113+0000 7f3d66000640 1 --2- 172.21.15.103:0/1205033955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d60109150 0x7f3d6010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.113+0000 7f3d66000640 1 --2- 172.21.15.103:0/1205033955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d601087d0 0x7f3d60108c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.113+0000 7f3d66000640 1 --2- 172.21.15.103:0/1205033955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d60102dc0 0x7f3d601031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.114 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.113+0000 7f3d66000640 1 -- 172.21.15.103:0/1205033955 >> 172.21.15.103:0/1205033955 conn(0x7f3d600fe830 msgr2=0x7f3d60100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.113+0000 7f3d66000640 1 -- 172.21.15.103:0/1205033955 shutdown_connections 2024-09-11T00:28:38.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.113+0000 7f3d66000640 1 -- 172.21.15.103:0/1205033955 wait complete. 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 Processor -- start 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 -- start start 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d60102dc0 0x7f3d600754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d601087d0 0x7f3d600759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d60109150 0x7f3d60075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112e60 con 0x7f3d60102dc0 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112ce0 con 0x7f3d601087d0 2024-09-11T00:28:38.116 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.114+0000 7f3d66000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112fe0 con 0x7f3d60109150 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d60102dc0 0x7f3d600754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d56a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d601087d0 0x7f3d600759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d60102dc0 0x7f3d600754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39706/0 (socket says 172.21.15.103:39706) 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 -- 172.21.15.103:0/526682522 learned_addr learned my addr 172.21.15.103:0/526682522 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5fe00640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d60109150 0x7f3d60075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 -- 172.21.15.103:0/526682522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d60109150 msgr2=0x7f3d60075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d60109150 0x7f3d60075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 -- 172.21.15.103:0/526682522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d601087d0 msgr2=0x7f3d600759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d601087d0 0x7f3d600759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.115+0000 7f3d5f400640 1 -- 172.21.15.103:0/526682522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d601a76f0 con 0x7f3d60102dc0 2024-09-11T00:28:38.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.116+0000 7f3d5f400640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d60102dc0 0x7f3d600754a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000e9f0 tx=0x7f3d5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.116+0000 7f3d5cc00640 1 -- 172.21.15.103:0/526682522 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5000ccc0 con 0x7f3d60102dc0 2024-09-11T00:28:38.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.116+0000 7f3d5cc00640 1 -- 172.21.15.103:0/526682522 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5000ce60 con 0x7f3d60102dc0 2024-09-11T00:28:38.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.116+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601a79e0 con 0x7f3d60102dc0 2024-09-11T00:28:38.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.116+0000 7f3d5cc00640 1 -- 172.21.15.103:0/526682522 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d50010430 con 0x7f3d60102dc0 2024-09-11T00:28:38.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.116+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601a7ef0 con 0x7f3d60102dc0 2024-09-11T00:28:38.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 Processor -- start 2024-09-11T00:28:38.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 -- start start 2024-09-11T00:28:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d5cc00640 1 -- 172.21.15.103:0/526682522 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f3d500046c0 con 0x7f3d60102dc0 2024-09-11T00:28:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414df63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e02360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e04c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c414d6d290 con 0x55c414df5ff0 2024-09-11T00:28:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c414d5c6c0 con 0x55c414df6910 2024-09-11T00:28:38.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.119+0000 7f3d68be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c414d604d0 con 0x55c414e028a0 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d5fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414df63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d56a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e04c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d5fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414df63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39714/0 (socket says 172.21.15.103:39714) 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d5fe00640 1 -- 172.21.15.103:0/3774579068 learned_addr learned my addr 172.21.15.103:0/3774579068 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d5f400640 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e02360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d56a00640 1 -- 172.21.15.103:0/3774579068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 msgr2=0x55c414e02360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d56a00640 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e02360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d56a00640 1 -- 172.21.15.103:0/3774579068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 msgr2=0x55c414df63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d56a00640 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414df63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.120+0000 7f3d56a00640 1 -- 172.21.15.103:0/3774579068 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c414ce9460 con 0x55c414e028a0 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.121+0000 7f3d56a00640 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e04c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3d4802a9e0 tx=0x7f3d48004190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=401430635a99edf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.121+0000 7f3d4fe00640 1 -- 172.21.15.103:0/3774579068 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48040070 con 0x55c414e028a0 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.121+0000 7f3d4fe00640 1 -- 172.21.15.103:0/3774579068 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d48005500 con 0x55c414e028a0 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.121+0000 7f3d4fe00640 1 -- 172.21.15.103:0/3774579068 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48035040 con 0x55c414e028a0 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.121+0000 7f3d5cc00640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d440b5360 0x7f3d440b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d5cc00640 1 -- 172.21.15.103:0/526682522 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f3d500e63b0 con 0x7f3d60102dc0 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d5fe00640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d440b5360 0x7f3d440b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 -- 172.21.15.103:0/3774579068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 msgr2=0x55c414e04c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e04c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3d4802a9e0 tx=0x7f3d48004190 comp rx=0 tx=0).stop 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 -- 172.21.15.103:0/3774579068 shutdown_connections 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e04c90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e02360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 --2- 172.21.15.103:0/3774579068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414df63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 -- 172.21.15.103:0/3774579068 >> 172.21.15.103:0/3774579068 conn(0x55c414df3e20 msgr2=0x55c414df4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 -- 172.21.15.103:0/3774579068 shutdown_connections 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d68be3740 1 -- 172.21.15.103:0/3774579068 wait complete. 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.122+0000 7f3d5fe00640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d440b5360 0x7f3d440b7820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d580049d0 tx=0x7f3d58009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 -- start start 2024-09-11T00:28:38.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 Processor -- start 2024-09-11T00:28:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414df5ff0 0x55c414dff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414dffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414e028a0 0x55c414e002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c414bbdbb0 con 0x55c414df6910 2024-09-11T00:28:38.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c414cf76c0 con 0x55c414e028a0 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.123+0000 7f3d68be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c414e054f0 con 0x55c414df5ff0 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d68be3740 10 client.0 _ll_register_callbacks cb 0x55c414d12490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d56a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414dffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d56a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414dffd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39722/0 (socket says 172.21.15.103:39722) 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5f400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414df5ff0 0x55c414dff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d56a00640 1 -- 172.21.15.103:0/1582873859 learned_addr learned my addr 172.21.15.103:0/1582873859 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5fe00640 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414e028a0 0x55c414e002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5f400640 1 -- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414e028a0 msgr2=0x55c414e002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5f400640 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414e028a0 0x55c414e002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5f400640 1 -- 172.21.15.103:0/1582873859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 msgr2=0x55c414dffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5f400640 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414dffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5f400640 1 -- 172.21.15.103:0/1582873859 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c414abab00 con 0x55c414df5ff0 2024-09-11T00:28:38.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d56a00640 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414dffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d5f400640 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414df5ff0 0x55c414dff830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55c414df9170 tx=0x7f3d500e9f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc5eae322685a4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.124+0000 7f3d4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1582873859 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d500ac290 con 0x55c414df5ff0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d48005500 con 0x55c414df5ff0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1582873859 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f3d500e6130 con 0x55c414df5ff0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1582873859 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3d500ace80 con 0x55c414df5ff0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d68be3740 10 client.26023 fetch_fsmap learned FSMap version 41 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c414b31960 con 0x55c414df5ff0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1582873859 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d48005500 con 0x55c414df5ff0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.125+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1582873859 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f3d500a8160 con 0x55c414df5ff0 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.126+0000 7f3d68be3740 10 client.26023 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.126+0000 7f3d68be3740 10 client.26023 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.126+0000 7f3d68be3740 20 client.26023 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.126+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e123e0 0x55c414e14880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.126+0000 7f3d68be3740 4 client.26023 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:38.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.126+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c414d6f1e0 con 0x55c414e123e0 2024-09-11T00:28:38.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.126+0000 7f3d56a00640 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e123e0 0x55c414e14880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.127+0000 7f3d56a00640 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e123e0 0x55c414e14880 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd61a9ef383483c6 server_cookie=55d4fdbfcd6bf824 in_seq=0 out_seq=0 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.127+0000 7f3d4cc00640 10 client.26023 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.127+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1582873859 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c414d6f1e0 con 0x55c414e123e0 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.127+0000 7f3d4cc00640 10 client.26023 handle_command_reply: tid=0 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 client.26023 shutdown 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414df5ff0 msgr2=0x55c414dff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414df5ff0 0x55c414dff830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55c414df9170 tx=0x7f3d500e9f80 comp rx=0 tx=0).stop 2024-09-11T00:28:38.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 shutdown_connections 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414e028a0 0x55c414e002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414dffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e123e0 0x55c414e14880 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1582873859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414df5ff0 0x55c414dff830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d4cc00640 0 client.26023 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 >> 172.21.15.103:0/1582873859 conn(0x55c414df3e20 msgr2=0x55c414df99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 shutdown_connections 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 1 -- 172.21.15.103:0/1582873859 wait complete. 2024-09-11T00:28:38.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.128+0000 7f3d68be3740 20 client.26023 trim_cache size 0 max 16384 2024-09-11T00:28:38.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.145+0000 7f3d68be3740 1 Processor -- start 2024-09-11T00:28:38.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d68be3740 1 -- start start 2024-09-11T00:28:38.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df5ff0 0x55c414df5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414e1a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.147 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e1a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d68be3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b4640 con 0x55c414df6910 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d68be3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b1eb0 con 0x55c414df5ff0 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d68be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601afd00 con 0x55c414e028a0 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d56a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414e1a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d56a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414e1a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39732/0 (socket says 172.21.15.103:39732) 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d56a00640 1 -- 172.21.15.103:0/2055087367 learned_addr learned my addr 172.21.15.103:0/2055087367 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d5fe00640 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e1a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.146+0000 7f3d5f400640 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df5ff0 0x55c414df5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d56a00640 1 -- 172.21.15.103:0/2055087367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 msgr2=0x55c414e1a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d56a00640 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e1a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d56a00640 1 -- 172.21.15.103:0/2055087367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df5ff0 msgr2=0x55c414df5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d56a00640 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df5ff0 0x55c414df5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d56a00640 1 -- 172.21.15.103:0/2055087367 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c414e14f00 con 0x55c414df6910 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d5f400640 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df5ff0 0x55c414df5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d56a00640 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414e1a190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009900 tx=0x7f3d48009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e40583b2bc26192 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d4f400640 1 -- 172.21.15.103:0/2055087367 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48040040 con 0x55c414df6910 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d4f400640 1 -- 172.21.15.103:0/2055087367 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d48004e50 con 0x55c414df6910 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.147+0000 7f3d4f400640 1 -- 172.21.15.103:0/2055087367 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d48035040 con 0x55c414df6910 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 -- 172.21.15.103:0/2055087367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 msgr2=0x55c414e1a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414e1a190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d48009900 tx=0x7f3d48009260 comp rx=0 tx=0).stop 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 -- 172.21.15.103:0/2055087367 shutdown_connections 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e1a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df6910 0x55c414e1a190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 --2- 172.21.15.103:0/2055087367 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df5ff0 0x55c414df5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 -- 172.21.15.103:0/2055087367 >> 172.21.15.103:0/2055087367 conn(0x55c414df3e20 msgr2=0x55c414df4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 -- 172.21.15.103:0/2055087367 shutdown_connections 2024-09-11T00:28:38.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 -- 172.21.15.103:0/2055087367 wait complete. 2024-09-11T00:28:38.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.148+0000 7f3d68be3740 1 -- start start 2024-09-11T00:28:38.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 1 Processor -- start 2024-09-11T00:28:38.153 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414e0acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e05770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b0370 con 0x55c414df5ff0 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b17e0 con 0x55c414df6910 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601b0a40 con 0x55c414e028a0 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d68be3740 10 client.0 _ll_register_callbacks cb 0x7f3d5000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d5f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414e0acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d5f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414e0acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39746/0 (socket says 172.21.15.103:39746) 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d5f400640 1 -- 172.21.15.103:0/1223565005 learned_addr learned my addr 172.21.15.103:0/1223565005 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d5fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.149+0000 7f3d56a00640 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e05770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d5f400640 1 -- 172.21.15.103:0/1223565005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 msgr2=0x55c414e05cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d5f400640 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e05cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d5f400640 1 -- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 msgr2=0x55c414e05770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d5f400640 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e05770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d5f400640 1 -- 172.21.15.103:0/1223565005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c414df48a0 con 0x55c414df5ff0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d5f400640 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414e0acb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d50012020 tx=0x7f3d500e99a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e0e18d5e6f0cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d4cc00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1223565005 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d500a8620 con 0x55c414df5ff0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1223565005 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f3d500abbe0 con 0x55c414df5ff0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d48004e50 con 0x55c414df5ff0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.150+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1223565005 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3d500ac0d0 con 0x55c414df5ff0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 10 client.15957 fetch_fsmap learned FSMap version 41 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c414e7ffb0 con 0x55c414df5ff0 2024-09-11T00:28:38.155 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1223565005 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d48004e50 con 0x55c414df5ff0 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1223565005 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f3d500adcc0 con 0x55c414df5ff0 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 10 client.15957 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 10 client.15957 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 20 client.15957 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e80780 0x55c414e82ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 4 client.15957 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "44154a20", "release": true, "await": true}] 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "set_id": "44154a20", "release": true, "await": true}) -- 0x55c414d6ac90 con 0x55c414e80780 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.151+0000 7f3d56a00640 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e80780 0x55c414e82ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.152+0000 7f3d56a00640 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e80780 0x55c414e82ba0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=458b2404cda804bb server_cookie=82925a2a2273df9e in_seq=0 out_seq=0 2024-09-11T00:28:38.156 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.152+0000 7f3d4cc00640 10 client.15957 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:38.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d4cc00640 1 -- 172.21.15.103:0/1223565005 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c414d6ac90 con 0x55c414e80780 2024-09-11T00:28:38.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d4cc00640 10 client.15957 handle_command_reply: tid=0 2024-09-11T00:28:38.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 client.15957 shutdown 2024-09-11T00:28:38.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 msgr2=0x55c414e0acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.301 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414e0acb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3d50012020 tx=0x7f3d500e99a0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 shutdown_connections 2024-09-11T00:28:38.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c414e80780 0x55c414e82ba0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c414e028a0 0x55c414e05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d4cc00640 0 client.15957 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:38.302 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c414df6910 0x55c414e05770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 --2- 172.21.15.103:0/1223565005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c414df5ff0 0x55c414e0acb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.300+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 >> 172.21.15.103:0/1223565005 conn(0x55c414df3e20 msgr2=0x55c414df91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.301+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 shutdown_connections 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.301+0000 7f3d68be3740 1 -- 172.21.15.103:0/1223565005 wait complete. 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.301+0000 7f3d68be3740 20 client.15957 trim_cache size 0 max 16384 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 41, 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:28:38.303 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 20, 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "44154a20": { 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 20, 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 150.0, 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:28:38.304 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:28:38.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d440b5360 msgr2=0x7f3d440b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d440b5360 0x7f3d440b7820 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3d580049d0 tx=0x7f3d58009600 comp rx=0 tx=0).stop 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d60102dc0 msgr2=0x7f3d600754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d60102dc0 0x7f3d600754a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d5000e9f0 tx=0x7f3d5000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 shutdown_connections 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f3d440b5360 0x7f3d440b7820 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3d60109150 0x7f3d60075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d601087d0 0x7f3d600759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 --2- 172.21.15.103:0/526682522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d60102dc0 0x7f3d600754a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.302+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 >> 172.21.15.103:0/526682522 conn(0x7f3d600fe830 msgr2=0x7f3d60100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.303+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 shutdown_connections 2024-09-11T00:28:38.306 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:28:38.303+0000 7f3d66000640 1 -- 172.21.15.103:0/526682522 wait complete. 2024-09-11T00:28:39.325 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '44154a20', total seconds elapsed: 34.4 2024-09-11T00:28:39.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:39 smithi103 ceph-mon[30578]: pgmap v567: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 45 KiB/s wr, 10 op/s 2024-09-11T00:28:39.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:39 smithi117 ceph-mon[35525]: pgmap v567: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 45 KiB/s wr, 10 op/s 2024-09-11T00:28:40.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:39 smithi183 ceph-mon[35934]: pgmap v567: 129 pgs: 129 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 45 KiB/s wr, 10 op/s 2024-09-11T00:28:41.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:41 smithi117 ceph-mon[35525]: pgmap v568: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 150 KiB/s wr, 27 op/s 2024-09-11T00:28:42.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:41 smithi183 ceph-mon[35934]: pgmap v568: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 150 KiB/s wr, 27 op/s 2024-09-11T00:28:42.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:41 smithi103 ceph-mon[30578]: pgmap v568: 129 pgs: 129 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 150 KiB/s wr, 27 op/s 2024-09-11T00:28:43.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:43 smithi117 ceph-mon[35525]: pgmap v569: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 123 KiB/s wr, 26 op/s 2024-09-11T00:28:43.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:43 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:28:43.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:43 smithi103 ceph-mon[30578]: pgmap v569: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 123 KiB/s wr, 26 op/s 2024-09-11T00:28:43.971 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:43 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:28:44.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:43 smithi183 ceph-mon[35934]: pgmap v569: 129 pgs: 129 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 123 KiB/s wr, 26 op/s 2024-09-11T00:28:44.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:43 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:28:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:44.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-09-11T00:28:44.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: Unable to set osd_memory_target on smithi103 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-09-11T00:28:44.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:28:44.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:28:44.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:28:45.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: Unable to set osd_memory_target on smithi103 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:28:45.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:28:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-11T00:28:45.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi103 to 176.4M 2024-09-11T00:28:45.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: Unable to set osd_memory_target on smithi103 to 185040486: error parsing value: Value '185040486' is below minimum 939524096 2024-09-11T00:28:45.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:28:45.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:28:45.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:28:46.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:45 smithi183 ceph-mon[35934]: pgmap v570: 129 pgs: 129 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 KiB/s wr, 160 op/s 2024-09-11T00:28:46.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:45 smithi103 ceph-mon[30578]: pgmap v570: 129 pgs: 129 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 KiB/s wr, 160 op/s 2024-09-11T00:28:46.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:45 smithi117 ceph-mon[35525]: pgmap v570: 129 pgs: 129 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 825 KiB/s wr, 160 op/s 2024-09-11T00:28:48.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:47 smithi183 ceph-mon[35934]: pgmap v571: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-09-11T00:28:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 -- 172.21.15.103:0/203215437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2520105ce0 msgr2=0x7f2520106160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 --2- 172.21.15.103:0/203215437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2520105ce0 0x7f2520106160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f250c009900 tx=0x7f250c02a3f0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 -- 172.21.15.103:0/203215437 shutdown_connections 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 --2- 172.21.15.103:0/203215437 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25201066a0 0x7f252010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 --2- 172.21.15.103:0/203215437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2520105ce0 0x7f2520106160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 --2- 172.21.15.103:0/203215437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 0x7f2520104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 -- 172.21.15.103:0/203215437 >> 172.21.15.103:0/203215437 conn(0x7f2520100470 msgr2=0x7f2520102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 -- 172.21.15.103:0/203215437 shutdown_connections 2024-09-11T00:28:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.142+0000 7f2525a00640 1 -- 172.21.15.103:0/203215437 wait complete. 2024-09-11T00:28:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.143+0000 7f2525a00640 1 Processor -- start 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.143+0000 7f2525a00640 1 -- start start 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f2525a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 0x7f252019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f2525a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2520105ce0 0x7f252019b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f2525a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25201066a0 0x7f252019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f2525a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2520110850 con 0x7f2520105ce0 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f2525a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25201106d0 con 0x7f25201066a0 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f2525a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25201109d0 con 0x7f2520104b90 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f251ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 0x7f252019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f251f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25201066a0 0x7f252019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f251ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 0x7f252019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57240/0 (socket says 172.21.15.103:57240) 2024-09-11T00:28:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f251ea00640 1 -- 172.21.15.103:0/1660849451 learned_addr learned my addr 172.21.15.103:0/1660849451 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f251f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25201066a0 0x7f252019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:50652/0 (socket says 172.21.15.103:50652) 2024-09-11T00:28:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.144+0000 7f251e000640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2520105ce0 0x7f252019b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f251ea00640 1 -- 172.21.15.103:0/1660849451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25201066a0 msgr2=0x7f252019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f251ea00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25201066a0 0x7f252019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f251ea00640 1 -- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2520105ce0 msgr2=0x7f252019b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f251ea00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2520105ce0 0x7f252019b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f251ea00640 1 -- 172.21.15.103:0/1660849451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f252019cab0 con 0x7f2520104b90 2024-09-11T00:28:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f251ea00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 0x7f252019af70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f251000eb90 tx=0x7f251000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f2517400640 1 -- 172.21.15.103:0/1660849451 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510004d20 con 0x7f2520104b90 2024-09-11T00:28:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f2517400640 1 -- 172.21.15.103:0/1660849451 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f251000ee90 con 0x7f2520104b90 2024-09-11T00:28:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252019cda0 con 0x7f2520104b90 2024-09-11T00:28:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.145+0000 7f2517400640 1 -- 172.21.15.103:0/1660849451 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510010720 con 0x7f2520104b90 2024-09-11T00:28:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.146+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25201a3470 con 0x7f2520104b90 2024-09-11T00:28:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.149+0000 7f25289c6740 1 Processor -- start 2024-09-11T00:28:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.149+0000 7f25289c6740 1 -- start start 2024-09-11T00:28:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f25289c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f25289c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b10c0 0x5570ff9b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f25289c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f25289c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570ff9216b0 con 0x5570ff9ae790 2024-09-11T00:28:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f25289c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570ff910ae0 con 0x5570ff9b10c0 2024-09-11T00:28:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f25289c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5570ff9148f0 con 0x5570ff9b3b00 2024-09-11T00:28:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f2517400640 1 -- 172.21.15.103:0/1660849451 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f2510005020 con 0x7f2520104b90 2024-09-11T00:28:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f251f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f251ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b10c0 0x5570ff9b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f251e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f251e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57256/0 (socket says 172.21.15.103:57256) 2024-09-11T00:28:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f251ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b10c0 0x5570ff9b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:50666/0 (socket says 172.21.15.103:50666) 2024-09-11T00:28:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f251f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57340/0 (socket says 172.21.15.103:57340) 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.150+0000 7f251e000640 1 -- 172.21.15.103:0/3957121348 learned_addr learned my addr 172.21.15.103:0/3957121348 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f251e000640 1 -- 172.21.15.103:0/3957121348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b10c0 msgr2=0x5570ff9b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f251e000640 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b10c0 0x5570ff9b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f251e000640 1 -- 172.21.15.103:0/3957121348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 msgr2=0x5570ff9b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f251e000640 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f251e000640 1 -- 172.21.15.103:0/3957121348 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ff8dd780 con 0x5570ff9b3b00 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f251ea00640 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b10c0 0x5570ff9b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f251e000640 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b5fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f250c005ec0 tx=0x7f250c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70307d073e3c347 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f250be00640 1 -- 172.21.15.103:0/3957121348 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c035030 con 0x5570ff9b3b00 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f250be00640 1 -- 172.21.15.103:0/3957121348 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c040070 con 0x5570ff9b3b00 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.151+0000 7f250be00640 1 -- 172.21.15.103:0/3957121348 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0318a0 con 0x5570ff9b3b00 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f2517400640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25040b5360 0x7f25040b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f2517400640 1 -- 172.21.15.103:0/1660849451 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f25100e65a0 con 0x7f2520104b90 2024-09-11T00:28:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f251f400640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25040b5360 0x7f25040b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 -- 172.21.15.103:0/3957121348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 msgr2=0x5570ff9b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b5fc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f250c005ec0 tx=0x7f250c004040 comp rx=0 tx=0).stop 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 -- 172.21.15.103:0/3957121348 shutdown_connections 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b5fc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b10c0 0x5570ff9b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 --2- 172.21.15.103:0/3957121348 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 -- 172.21.15.103:0/3957121348 >> 172.21.15.103:0/3957121348 conn(0x5570ff9a98e0 msgr2=0x5570ff9aa670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 -- 172.21.15.103:0/3957121348 shutdown_connections 2024-09-11T00:28:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.152+0000 7f25289c6740 1 -- 172.21.15.103:0/3957121348 wait complete. 2024-09-11T00:28:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.153+0000 7f251f400640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25040b5360 0x7f25040b7820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f252019bcf0 tx=0x7f2518005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.153+0000 7f25289c6740 1 -- start start 2024-09-11T00:28:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.153+0000 7f25289c6740 1 Processor -- start 2024-09-11T00:28:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f25289c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9ae790 0x5570ff9b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f25289c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9b10c0 0x5570ff9b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f25289c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f25289c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5570ff848ab0 con 0x5570ff9b10c0 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f25289c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5570ff8b7570 con 0x5570ff9ae790 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f25289c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5570ff9b6a80 con 0x5570ff9b3b00 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f25289c6740 10 client.0 _ll_register_callbacks cb 0x5570ff8c6100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9ae790 0x5570ff9b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9ae790 0x5570ff9b7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:50668/0 (socket says 172.21.15.103:50668) 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 -- 172.21.15.103:0/210373761 learned_addr learned my addr 172.21.15.103:0/210373761 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251e000640 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9b10c0 0x5570ff9b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251f400640 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 -- 172.21.15.103:0/210373761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 msgr2=0x5570ff9b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 -- 172.21.15.103:0/210373761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9b10c0 msgr2=0x5570ff9b84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9b10c0 0x5570ff9b84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 -- 172.21.15.103:0/210373761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ff672790 con 0x5570ff9ae790 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f251ea00640 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9ae790 0x5570ff9b7e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25100049d0 tx=0x7f25100096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a80ee2a2324f4a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.154+0000 7f2508c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f2508c00640 1 -- 172.21.15.103:0/210373761 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25100a8870 con 0x5570ff9ae790 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f2508c00640 1 -- 172.21.15.103:0/210373761 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f25100ace40 con 0x5570ff9ae790 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f250c040070 con 0x5570ff9ae790 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f2508c00640 1 -- 172.21.15.103:0/210373761 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f25100a8140 con 0x5570ff9ae790 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 10 client.26015 fetch_fsmap learned FSMap version 41 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570ff6eb620 con 0x5570ff9ae790 2024-09-11T00:28:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f2508c00640 1 -- 172.21.15.103:0/210373761 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c040070 con 0x5570ff9ae790 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f2508c00640 1 -- 172.21.15.103:0/210373761 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f25100b5ad0 con 0x5570ff9ae790 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 10 client.26015 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 20 client.26015 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ff9c3750 0x5570ff9c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 4 client.26015 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.155+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570ff923600 con 0x5570ff9c3750 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.156+0000 7f251e000640 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ff9c3750 0x5570ff9c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.156+0000 7f251e000640 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ff9c3750 0x5570ff9c5bd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e665449e6b8054a server_cookie=abd4a3bdfb444c5e in_seq=0 out_seq=0 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.156+0000 7f2508c00640 10 client.26015 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.157+0000 7f2508c00640 1 -- 172.21.15.103:0/210373761 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5570ff923600 con 0x5570ff9c3750 2024-09-11T00:28:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.157+0000 7f2508c00640 10 client.26015 handle_command_reply: tid=0 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.157+0000 7f25289c6740 1 client.26015 shutdown 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9ae790 msgr2=0x5570ff9b7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9ae790 0x5570ff9b7e10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f25100049d0 tx=0x7f25100096c0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 shutdown_connections 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ff9c3750 0x5570ff9c5bd0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b3b00 0x5570ff9b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9b10c0 0x5570ff9b84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 --2- 172.21.15.103:0/210373761 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9ae790 0x5570ff9b7e10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 >> 172.21.15.103:0/210373761 conn(0x5570ff9a98e0 msgr2=0x5570ff9a8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f2508c00640 0 client.26015 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 shutdown_connections 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 1 -- 172.21.15.103:0/210373761 wait complete. 2024-09-11T00:28:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.158+0000 7f25289c6740 20 client.26015 trim_cache size 0 max 16384 2024-09-11T00:28:48.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:47 smithi103 ceph-mon[30578]: pgmap v571: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-09-11T00:28:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.176+0000 7f25289c6740 1 Processor -- start 2024-09-11T00:28:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f25289c6740 1 -- start start 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f25289c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f25289c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9b7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f25289c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f25289c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25201b0120 con 0x5570ff9ae790 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f25289c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25201ad990 con 0x5570ff9b3b00 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f25289c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25201ab7e0 con 0x5570ff9b10c0 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f251f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f251ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f251f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:50670/0 (socket says 172.21.15.103:50670) 2024-09-11T00:28:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f251e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9b7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.177+0000 7f251f400640 1 -- 172.21.15.103:0/1468361930 learned_addr learned my addr 172.21.15.103:0/1468361930 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.178+0000 7f251ea00640 1 -- 172.21.15.103:0/1468361930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 msgr2=0x5570ff9b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.178+0000 7f251ea00640 1 --2- 172.21.15.103:0/1468361930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9b7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.178+0000 7f251ea00640 1 -- 172.21.15.103:0/1468361930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 msgr2=0x5570ff9b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.178+0000 7f251ea00640 1 --2- 172.21.15.103:0/1468361930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.178+0000 7f251ea00640 1 -- 172.21.15.103:0/1468361930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ff9b0ef0 con 0x5570ff9ae790 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.178+0000 7f251ea00640 1 --2- 172.21.15.103:0/1468361930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b6e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f25100ac6d0 tx=0x7f251000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c0de69d190effe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.178+0000 7f250b400640 1 -- 172.21.15.103:0/1468361930 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25100c6030 con 0x5570ff9ae790 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.179+0000 7f250b400640 1 -- 172.21.15.103:0/1468361930 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25100ac4e0 con 0x5570ff9ae790 2024-09-11T00:28:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.179+0000 7f250b400640 1 -- 172.21.15.103:0/1468361930 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25100ac140 con 0x5570ff9ae790 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.179+0000 7f25289c6740 1 -- 172.21.15.103:0/1468361930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 msgr2=0x5570ff9b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.179+0000 7f25289c6740 1 --2- 172.21.15.103:0/1468361930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b6e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f25100ac6d0 tx=0x7f251000cf90 comp rx=0 tx=0).stop 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 -- 172.21.15.103:0/1468361930 shutdown_connections 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 --2- 172.21.15.103:0/1468361930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 --2- 172.21.15.103:0/1468361930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9b7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 --2- 172.21.15.103:0/1468361930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9b6e50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 -- 172.21.15.103:0/1468361930 >> 172.21.15.103:0/1468361930 conn(0x5570ff9a98e0 msgr2=0x5570ff9c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 -- 172.21.15.103:0/1468361930 shutdown_connections 2024-09-11T00:28:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 -- 172.21.15.103:0/1468361930 wait complete. 2024-09-11T00:28:48.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.180+0000 7f25289c6740 1 -- start start 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 1 Processor -- start 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9be060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25201abe50 con 0x5570ff9ae790 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25201ad2c0 con 0x5570ff9b3b00 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25201ac520 con 0x5570ff9b10c0 2024-09-11T00:28:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f25289c6740 10 client.0 _ll_register_callbacks cb 0x7f2510004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f251e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f251f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9be060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f251e000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9bd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57304/0 (socket says 172.21.15.103:57304) 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.181+0000 7f251e000640 1 -- 172.21.15.103:0/651685743 learned_addr learned my addr 172.21.15.103:0/651685743 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251ea00640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251e000640 1 -- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 msgr2=0x5570ff9be060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251e000640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9be060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251e000640 1 -- 172.21.15.103:0/651685743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 msgr2=0x5570ff9bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251e000640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251e000640 1 -- 172.21.15.103:0/651685743 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ff9be890 con 0x5570ff9b10c0 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251f400640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9be060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:28:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251ea00640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f251e000640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9bd490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f250c005740 tx=0x7f250c0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25bf5f27d72e354 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f2508c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.182+0000 7f2508c00640 1 -- 172.21.15.103:0/651685743 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c035040 con 0x5570ff9b10c0 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.183+0000 7f2508c00640 1 -- 172.21.15.103:0/651685743 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f250c0037f0 con 0x5570ff9b10c0 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.183+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25100ac4e0 con 0x5570ff9b10c0 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.183+0000 7f2508c00640 1 -- 172.21.15.103:0/651685743 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f250c062050 con 0x5570ff9b10c0 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.183+0000 7f25289c6740 10 client.26047 fetch_fsmap learned FSMap version 41 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.183+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570ff9bfd00 con 0x5570ff9b10c0 2024-09-11T00:28:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.183+0000 7f2508c00640 1 -- 172.21.15.103:0/651685743 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25100ac4e0 con 0x5570ff9b10c0 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f2508c00640 1 -- 172.21.15.103:0/651685743 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f250c031aa0 con 0x5570ff9b10c0 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f25289c6740 10 client.26047 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f25289c6740 10 client.26047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f25289c6740 20 client.26047 populate_metadata read hostname 'smithi103' 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f25289c6740 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ffa38fc0 0x5570ffa3b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f25289c6740 4 client.26047 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5570ff91f0b0 con 0x5570ffa38fc0 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.184+0000 7f251ea00640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ffa38fc0 0x5570ffa3b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.185+0000 7f2508c00640 10 client.26047 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.185+0000 7f251ea00640 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ffa38fc0 0x5570ffa3b3b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d2fa716f26cf26 server_cookie=16c9344f0bede4dc in_seq=0 out_seq=0 2024-09-11T00:28:48.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:47 smithi117 ceph-mon[35525]: pgmap v571: 129 pgs: 129 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 209 op/s 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.196+0000 7f2508c00640 1 -- 172.21.15.103:0/651685743 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5570ff91f0b0 con 0x5570ffa38fc0 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.196+0000 7f2508c00640 10 client.26047 handle_command_reply: tid=0 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 client.26047 shutdown 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 msgr2=0x5570ff9bd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9bd490 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f250c005740 tx=0x7f250c0058e0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 shutdown_connections 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5570ff9b3b00 0x5570ff9be060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5570ffa38fc0 0x5570ffa3b3b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5570ff9b10c0 0x5570ff9bd490 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 --2- 172.21.15.103:0/651685743 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5570ff9ae790 0x5570ff9bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f2508c00640 0 client.26047 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 >> 172.21.15.103:0/651685743 conn(0x5570ff9a98e0 msgr2=0x5570ff9b6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 shutdown_connections 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 1 -- 172.21.15.103:0/651685743 wait complete. 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.197+0000 7f25289c6740 20 client.26047 trim_cache size 0 max 16384 2024-09-11T00:28:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:28:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:28:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:28:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:28:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.198+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25040b5360 msgr2=0x7f25040b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.198+0000 7f2525a00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25040b5360 0x7f25040b7820 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f252019bcf0 tx=0x7f2518005f90 comp rx=0 tx=0).stop 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.198+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 msgr2=0x7f252019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.198+0000 7f2525a00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 0x7f252019af70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f251000eb90 tx=0x7f251000c6a0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 shutdown_connections 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25040b5360 0x7f25040b7820 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25201066a0 0x7f252019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2520105ce0 0x7f252019b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 --2- 172.21.15.103:0/1660849451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2520104b90 0x7f252019af70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 >> 172.21.15.103:0/1660849451 conn(0x7f2520100470 msgr2=0x7f25201090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 shutdown_connections 2024-09-11T00:28:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:28:48.199+0000 7f2525a00640 1 -- 172.21.15.103:0/1660849451 wait complete. 2024-09-11T00:28:50.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:49 smithi183 ceph-mon[35934]: pgmap v572: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 309 op/s 2024-09-11T00:28:50.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:49 smithi103 ceph-mon[30578]: pgmap v572: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 309 op/s 2024-09-11T00:28:50.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:49 smithi117 ceph-mon[35525]: pgmap v572: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.6 MiB/s wr, 309 op/s 2024-09-11T00:28:52.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:51 smithi183 ceph-mon[35934]: pgmap v573: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 383 op/s 2024-09-11T00:28:52.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:51 smithi103 ceph-mon[30578]: pgmap v573: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 383 op/s 2024-09-11T00:28:52.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:51 smithi117 ceph-mon[35525]: pgmap v573: 129 pgs: 129 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 383 op/s 2024-09-11T00:28:54.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:53 smithi183 ceph-mon[35934]: pgmap v574: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-11T00:28:54.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:53 smithi103 ceph-mon[30578]: pgmap v574: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-11T00:28:54.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:53 smithi117 ceph-mon[35525]: pgmap v574: 129 pgs: 129 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.0 MiB/s wr, 366 op/s 2024-09-11T00:28:54.491 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0x9ff773 2024-09-11T00:28:55.202 INFO:tasks.workunit.client.0.smithi103.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-09-11T00:28:56.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:55 smithi183 ceph-mon[35934]: pgmap v575: 129 pgs: 129 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.6 MiB/s wr, 494 op/s 2024-09-11T00:28:56.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:55 smithi103 ceph-mon[30578]: pgmap v575: 129 pgs: 129 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.6 MiB/s wr, 494 op/s 2024-09-11T00:28:56.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:55 smithi117 ceph-mon[35525]: pgmap v575: 129 pgs: 129 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.6 MiB/s wr, 494 op/s 2024-09-11T00:28:58.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:57 smithi183 ceph-mon[35934]: pgmap v576: 129 pgs: 129 active+clean; 61 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.2 MiB/s wr, 401 op/s 2024-09-11T00:28:58.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:57 smithi103 ceph-mon[30578]: pgmap v576: 129 pgs: 129 active+clean; 61 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.2 MiB/s wr, 401 op/s 2024-09-11T00:28:58.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:57 smithi117 ceph-mon[35525]: pgmap v576: 129 pgs: 129 active+clean; 61 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.2 MiB/s wr, 401 op/s 2024-09-11T00:29:00.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:28:59 smithi183 ceph-mon[35934]: pgmap v577: 129 pgs: 129 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 446 op/s 2024-09-11T00:29:00.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:28:59 smithi103 ceph-mon[30578]: pgmap v577: 129 pgs: 129 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 446 op/s 2024-09-11T00:29:00.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:28:59 smithi117 ceph-mon[35525]: pgmap v577: 129 pgs: 129 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 446 op/s 2024-09-11T00:29:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.337+0000 7f7e1a800640 1 -- 172.21.15.103:0/3090474821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 msgr2=0x7f7e14102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.337+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3090474821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 0x7f7e14102b90 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009930 tx=0x7f7e0802a440 comp rx=0 tx=0).stop 2024-09-11T00:29:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.338+0000 7f7e1a800640 1 -- 172.21.15.103:0/3090474821 shutdown_connections 2024-09-11T00:29:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.338+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3090474821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141042a0 0x7f7e1410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.338+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3090474821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e141038e0 0x7f7e14103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.338+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3090474821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 0x7f7e14102b90 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.338+0000 7f7e1a800640 1 -- 172.21.15.103:0/3090474821 >> 172.21.15.103:0/3090474821 conn(0x7f7e140fe030 msgr2=0x7f7e14100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.338+0000 7f7e1a800640 1 -- 172.21.15.103:0/3090474821 shutdown_connections 2024-09-11T00:29:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.338+0000 7f7e1a800640 1 -- 172.21.15.103:0/3090474821 wait complete. 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.339+0000 7f7e1a800640 1 Processor -- start 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.339+0000 7f7e1a800640 1 -- start start 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.339+0000 7f7e1a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 0x7f7e141050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e1a800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141038e0 0x7f7e14105730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e1a800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e141042a0 0x7f7e14106360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e1a800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1410e450 con 0x7f7e14102790 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e1a800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1410e2d0 con 0x7f7e141042a0 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e1a800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1410e5d0 con 0x7f7e141038e0 2024-09-11T00:29:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141038e0 0x7f7e14105730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e19e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e141042a0 0x7f7e14106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141038e0 0x7f7e14105730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:50418/0 (socket says 172.21.15.103:50418) 2024-09-11T00:29:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 -- 172.21.15.103:0/3319331333 learned_addr learned my addr 172.21.15.103:0/3319331333 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e19400640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 0x7f7e141050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 -- 172.21.15.103:0/3319331333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e141042a0 msgr2=0x7f7e14106360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e141042a0 0x7f7e14106360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 -- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 msgr2=0x7f7e141050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 0x7f7e141050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e18a00640 1 -- 172.21.15.103:0/3319331333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e14106c20 con 0x7f7e141038e0 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.340+0000 7f7e19400640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 0x7f7e141050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.341+0000 7f7e18a00640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141038e0 0x7f7e14105730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000ea60 tx=0x7f7e0000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.341+0000 7f7e12000640 1 -- 172.21.15.103:0/3319331333 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000cde0 con 0x7f7e141038e0 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.341+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e14106f10 con 0x7f7e141038e0 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.341+0000 7f7e12000640 1 -- 172.21.15.103:0/3319331333 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e000035f0 con 0x7f7e141038e0 2024-09-11T00:29:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.341+0000 7f7e12000640 1 -- 172.21.15.103:0/3319331333 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00010690 con 0x7f7e141038e0 2024-09-11T00:29:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.341+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e141075b0 con 0x7f7e141038e0 2024-09-11T00:29:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 Processor -- start 2024-09-11T00:29:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 -- start start 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e37dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e380690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 0x55d35e3831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d35e2ee6b0 con 0x55d35e37e2a0 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d35e2ddae0 con 0x55d35e37b970 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.345+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d35e2e18f0 con 0x55d35e380ce0 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e37dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e37dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:60558/0 (socket says 172.21.15.103:60558) 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 -- 172.21.15.103:0/2557419387 learned_addr learned my addr 172.21.15.103:0/2557419387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19400640 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e380690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e12000640 1 -- 172.21.15.103:0/3319331333 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7e00004000 con 0x7f7e141038e0 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 -- 172.21.15.103:0/2557419387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 msgr2=0x55d35e3831a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 0x55d35e3831a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 -- 172.21.15.103:0/2557419387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 msgr2=0x55d35e380690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e380690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 -- 172.21.15.103:0/2557419387 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d35e2aa780 con 0x55d35e37b970 2024-09-11T00:29:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.346+0000 7f7e19e00640 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e37dd60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009f90 tx=0x7f7e0c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7b3848d56e1a84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.347+0000 7f7e06600640 1 -- 172.21.15.103:0/2557419387 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c004d20 con 0x55d35e37b970 2024-09-11T00:29:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.347+0000 7f7e06600640 1 -- 172.21.15.103:0/2557419387 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0c004050 con 0x55d35e37b970 2024-09-11T00:29:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.347+0000 7f7e06600640 1 -- 172.21.15.103:0/2557419387 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c005280 con 0x55d35e37b970 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.347+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2557419387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 msgr2=0x55d35e37dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.347+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e37dd60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009f90 tx=0x7f7e0c0118d0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2557419387 shutdown_connections 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 0x55d35e3831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e380690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2557419387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e37dd60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2557419387 >> 172.21.15.103:0/2557419387 conn(0x55d35e376ac0 msgr2=0x55d35e377850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2557419387 shutdown_connections 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2557419387 wait complete. 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e12000640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7dfc0b51f0 0x7f7dfc0b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e12000640 1 -- 172.21.15.103:0/3319331333 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f7e000e62f0 con 0x7f7e141038e0 2024-09-11T00:29:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 -- start start 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e19400640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7dfc0b51f0 0x7f7dfc0b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.348+0000 7f7e1ec53740 1 Processor -- start 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e385110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e3857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 0x55d35e386380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d35e215ab0 con 0x55d35e37e2a0 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55d35e284570 con 0x55d35e37b970 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d35e383cc0 con 0x55d35e380ce0 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e1ec53740 10 client.0 _ll_register_callbacks cb 0x55d35e293100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e385110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e385110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:60566/0 (socket says 172.21.15.103:60566) 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 -- 172.21.15.103:0/2842162097 learned_addr learned my addr 172.21.15.103:0/2842162097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e18a00640 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e3857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7dfc0b51f0 0x7f7dfc0b76b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e08006010 tx=0x7f7e08005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 -- 172.21.15.103:0/2842162097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 msgr2=0x55d35e386380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 0x55d35e386380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 -- 172.21.15.103:0/2842162097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 msgr2=0x55d35e3857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e3857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.349+0000 7f7e19400640 1 -- 172.21.15.103:0/2842162097 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d35e03f790 con 0x55d35e37b970 2024-09-11T00:29:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7e19400640 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e385110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009930 tx=0x7f7e08009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f300a4e79295ecd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7dfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:29:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2842162097 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08035040 con 0x55d35e37b970 2024-09-11T00:29:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0c004050 con 0x55d35e37b970 2024-09-11T00:29:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2842162097 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f7e08004d10 con 0x55d35e37b970 2024-09-11T00:29:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2842162097 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7e08059050 con 0x55d35e37b970 2024-09-11T00:29:01.359 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7e1ec53740 10 client.26033 fetch_fsmap learned FSMap version 41 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.350+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d35e0b8620 con 0x55d35e37b970 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2842162097 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0c004050 con 0x55d35e37b970 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2842162097 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f7e080328e0 con 0x55d35e37b970 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e1ec53740 10 client.26033 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e1ec53740 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e1ec53740 20 client.26033 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e390b00 0x55d35e392f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e1ec53740 4 client.26033 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d35e2f0600 con 0x55d35e390b00 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e18a00640 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e390b00 0x55d35e392f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7e18a00640 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e390b00 0x55d35e392f80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e60670d3ca08ce server_cookie=786827ce1a7baf4c in_seq=0 out_seq=0 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.351+0000 7f7dfaa00640 10 client.26033 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.352+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2842162097 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d35e2f0600 con 0x55d35e390b00 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.352+0000 7f7dfaa00640 10 client.26033 handle_command_reply: tid=0 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.352+0000 7f7e1ec53740 1 client.26033 shutdown 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 msgr2=0x55d35e385110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e385110 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009930 tx=0x7f7e08009260 comp rx=0 tx=0).stop 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 shutdown_connections 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e390b00 0x55d35e392f80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e380ce0 0x55d35e386380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e3857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2842162097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e385110 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 >> 172.21.15.103:0/2842162097 conn(0x55d35e376ac0 msgr2=0x55d35e375bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7dfaa00640 0 client.26033 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 shutdown_connections 2024-09-11T00:29:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2842162097 wait complete. 2024-09-11T00:29:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.353+0000 7f7e1ec53740 20 client.26033 trim_cache size 0 max 16384 2024-09-11T00:29:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.372+0000 7f7e1ec53740 1 Processor -- start 2024-09-11T00:29:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.372+0000 7f7e1ec53740 1 -- start start 2024-09-11T00:29:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37b970 0x55d35e37aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e37b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e380ce0 0x55d35e384090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141b4450 con 0x55d35e37e2a0 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141b1cc0 con 0x55d35e380ce0 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141afb10 con 0x55d35e37b970 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e19400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37b970 0x55d35e37aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e18a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e37b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e19400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37b970 0x55d35e37aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:50434/0 (socket says 172.21.15.103:50434) 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e19400640 1 -- 172.21.15.103:0/25286607 learned_addr learned my addr 172.21.15.103:0/25286607 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.373+0000 7f7e19e00640 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e380ce0 0x55d35e384090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e19400640 1 -- 172.21.15.103:0/25286607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e380ce0 msgr2=0x55d35e384090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e19400640 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e380ce0 0x55d35e384090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e19400640 1 -- 172.21.15.103:0/25286607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 msgr2=0x55d35e37b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e19400640 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e37b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e19400640 1 -- 172.21.15.103:0/25286607 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d35e384c10 con 0x55d35e37b970 2024-09-11T00:29:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e18a00640 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e37b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e19400640 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37b970 0x55d35e37aef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e08032d80 tx=0x7f7e0802ab00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb499be2ce6ce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e05c00640 1 -- 172.21.15.103:0/25286607 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e08035070 con 0x55d35e37b970 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e05c00640 1 -- 172.21.15.103:0/25286607 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e08032a60 con 0x55d35e37b970 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.374+0000 7f7e05c00640 1 -- 172.21.15.103:0/25286607 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0803cb90 con 0x55d35e37b970 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 -- 172.21.15.103:0/25286607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37b970 msgr2=0x55d35e37aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37b970 0x55d35e37aef0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e08032d80 tx=0x7f7e0802ab00 comp rx=0 tx=0).stop 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 -- 172.21.15.103:0/25286607 shutdown_connections 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e380ce0 0x55d35e384090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e37e2a0 0x55d35e37b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/25286607 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37b970 0x55d35e37aef0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 -- 172.21.15.103:0/25286607 >> 172.21.15.103:0/25286607 conn(0x55d35e376ac0 msgr2=0x55d35e377780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 -- 172.21.15.103:0/25286607 shutdown_connections 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.375+0000 7f7e1ec53740 1 -- 172.21.15.103:0/25286607 wait complete. 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 -- start start 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 Processor -- start 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e38cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37e2a0 0x55d35e38bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e380ce0 0x55d35e38c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141b0180 con 0x55d35e380ce0 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141b15f0 con 0x55d35e37b970 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e141b0850 con 0x55d35e37e2a0 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e1ec53740 10 client.0 _ll_register_callbacks cb 0x7f7e0c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e18a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37e2a0 0x55d35e38bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.376+0000 7f7e19400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e38cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37e2a0 0x55d35e38bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:50446/0 (socket says 172.21.15.103:50446) 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 -- 172.21.15.103:0/2782757094 learned_addr learned my addr 172.21.15.103:0/2782757094 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e19e00640 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e380ce0 0x55d35e38c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 -- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 msgr2=0x55d35e38cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e38cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 -- 172.21.15.103:0/2782757094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e380ce0 msgr2=0x55d35e38c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e380ce0 0x55d35e38c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 -- 172.21.15.103:0/2782757094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d35e38a610 con 0x55d35e37e2a0 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e19400640 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e38cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e18a00640 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37e2a0 0x55d35e38bb30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e000a8cf0 tx=0x7f7e0000c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db8224d38c7a5149 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7dfaa00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:29:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2782757094 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e000ac720 con 0x55d35e37e2a0 2024-09-11T00:29:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2782757094 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f7e000037a0 con 0x55d35e37e2a0 2024-09-11T00:29:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.377+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e08032a60 con 0x55d35e37e2a0 2024-09-11T00:29:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2782757094 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7e000b1200 con 0x55d35e37e2a0 2024-09-11T00:29:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 10 client.26059 fetch_fsmap learned FSMap version 41 2024-09-11T00:29:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d35e3fddb0 con 0x55d35e37e2a0 2024-09-11T00:29:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2782757094 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e08032a60 con 0x55d35e37e2a0 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2782757094 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f7e000c4030 con 0x55d35e37e2a0 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 10 client.26059 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 10 client.26059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 20 client.26059 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e3fe7a0 0x55d35e400bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 4 client.26059 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d35e2ec0b0 con 0x55d35e3fe7a0 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.378+0000 7f7e19400640 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e3fe7a0 0x55d35e400bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.379+0000 7f7dfaa00640 10 client.26059 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.379+0000 7f7e19400640 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e3fe7a0 0x55d35e400bc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621d970600d7669f server_cookie=4c103b184aefb18e in_seq=0 out_seq=0 2024-09-11T00:29:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.395+0000 7f7dfaa00640 1 -- 172.21.15.103:0/2782757094 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d35e2ec0b0 con 0x55d35e3fe7a0 2024-09-11T00:29:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.395+0000 7f7dfaa00640 10 client.26059 handle_command_reply: tid=0 2024-09-11T00:29:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.395+0000 7f7e1ec53740 1 client.26059 shutdown 2024-09-11T00:29:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.395+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37e2a0 msgr2=0x55d35e38bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.395+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37e2a0 0x55d35e38bb30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e000a8cf0 tx=0x7f7e0000c890 comp rx=0 tx=0).stop 2024-09-11T00:29:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 shutdown_connections 2024-09-11T00:29:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d35e380ce0 0x55d35e38c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55d35e3fe7a0 0x55d35e400bc0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d35e37e2a0 0x55d35e38bb30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7dfaa00640 0 client.26059 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 --2- 172.21.15.103:0/2782757094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55d35e37b970 0x55d35e38cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 >> 172.21.15.103:0/2782757094 conn(0x55d35e376ac0 msgr2=0x55d35e384090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 shutdown_connections 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 1 -- 172.21.15.103:0/2782757094 wait complete. 2024-09-11T00:29:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.396+0000 7f7e1ec53740 20 client.26059 trim_cache size 0 max 16384 2024-09-11T00:29:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:29:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:29:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:29:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7dfc0b51f0 msgr2=0x7f7dfc0b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7dfc0b51f0 0x7f7dfc0b76b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e08006010 tx=0x7f7e08005b50 comp rx=0 tx=0).stop 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141038e0 msgr2=0x7f7e14105730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141038e0 0x7f7e14105730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000ea60 tx=0x7f7e0000ef20 comp rx=0 tx=0).stop 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 shutdown_connections 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7dfc0b51f0 0x7f7dfc0b76b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e141042a0 0x7f7e14106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e141038e0 0x7f7e14105730 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 --2- 172.21.15.103:0/3319331333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e14102790 0x7f7e141050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 >> 172.21.15.103:0/3319331333 conn(0x7f7e140fe030 msgr2=0x7f7e14075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 shutdown_connections 2024-09-11T00:29:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:01.397+0000 7f7e1a800640 1 -- 172.21.15.103:0/3319331333 wait complete. 2024-09-11T00:29:02.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:01 smithi183 ceph-mon[35934]: pgmap v578: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 419 op/s 2024-09-11T00:29:02.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:01 smithi103 ceph-mon[30578]: pgmap v578: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 419 op/s 2024-09-11T00:29:02.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:01 smithi117 ceph-mon[35525]: pgmap v578: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 419 op/s 2024-09-11T00:29:04.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:03 smithi183 ceph-mon[35934]: pgmap v579: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 342 op/s 2024-09-11T00:29:04.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:03 smithi103 ceph-mon[30578]: pgmap v579: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 342 op/s 2024-09-11T00:29:04.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:03 smithi117 ceph-mon[35525]: pgmap v579: 129 pgs: 129 active+clean; 65 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 342 op/s 2024-09-11T00:29:06.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:05 smithi183 ceph-mon[35934]: pgmap v580: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 468 op/s 2024-09-11T00:29:06.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:05 smithi103 ceph-mon[30578]: pgmap v580: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 468 op/s 2024-09-11T00:29:06.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:05 smithi117 ceph-mon[35525]: pgmap v580: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 468 op/s 2024-09-11T00:29:08.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:07 smithi183 ceph-mon[35934]: pgmap v581: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 380 op/s 2024-09-11T00:29:08.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:07 smithi103 ceph-mon[30578]: pgmap v581: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 380 op/s 2024-09-11T00:29:08.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:07 smithi117 ceph-mon[35525]: pgmap v581: 129 pgs: 129 active+clean; 63 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 380 op/s 2024-09-11T00:29:08.967 INFO:tasks.workunit.client.0.smithi103.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-09-11T00:29:10.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:09 smithi183 ceph-mon[35934]: pgmap v582: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 433 op/s 2024-09-11T00:29:10.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:09 smithi103 ceph-mon[30578]: pgmap v582: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 433 op/s 2024-09-11T00:29:10.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:09 smithi117 ceph-mon[35525]: pgmap v582: 129 pgs: 129 active+clean; 60 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 433 op/s 2024-09-11T00:29:12.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:11 smithi183 ceph-mon[35934]: pgmap v583: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.3 MiB/s wr, 401 op/s 2024-09-11T00:29:12.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:11 smithi103 ceph-mon[30578]: pgmap v583: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.3 MiB/s wr, 401 op/s 2024-09-11T00:29:12.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:11 smithi117 ceph-mon[35525]: pgmap v583: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.3 MiB/s wr, 401 op/s 2024-09-11T00:29:13.987 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:13 smithi103 ceph-mon[30578]: pgmap v584: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 327 op/s 2024-09-11T00:29:13.988 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:13 smithi117 ceph-mon[35525]: pgmap v584: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 327 op/s 2024-09-11T00:29:14.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:13 smithi183 ceph-mon[35934]: pgmap v584: 129 pgs: 129 active+clean; 64 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 327 op/s 2024-09-11T00:29:14.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.535+0000 7f6d41800640 1 -- 172.21.15.103:0/3730459816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c075ac0 msgr2=0x7f6d3c075f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.535+0000 7f6d41800640 1 --2- 172.21.15.103:0/3730459816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c075ac0 0x7f6d3c075f40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009900 tx=0x7f6d2802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.536+0000 7f6d41800640 1 -- 172.21.15.103:0/3730459816 shutdown_connections 2024-09-11T00:29:14.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.536+0000 7f6d41800640 1 --2- 172.21.15.103:0/3730459816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c10a910 0x7f6d3c10cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.536+0000 7f6d41800640 1 --2- 172.21.15.103:0/3730459816 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c075ac0 0x7f6d3c075f40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.538 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.536+0000 7f6d41800640 1 --2- 172.21.15.103:0/3730459816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c076f30 0x7f6d3c075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.536+0000 7f6d41800640 1 -- 172.21.15.103:0/3730459816 >> 172.21.15.103:0/3730459816 conn(0x7f6d3c0fe250 msgr2=0x7f6d3c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.536+0000 7f6d41800640 1 -- 172.21.15.103:0/3730459816 shutdown_connections 2024-09-11T00:29:14.539 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.536+0000 7f6d41800640 1 -- 172.21.15.103:0/3730459816 wait complete. 2024-09-11T00:29:14.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.537+0000 7f6d41800640 1 Processor -- start 2024-09-11T00:29:14.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.537+0000 7f6d41800640 1 -- start start 2024-09-11T00:29:14.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d41800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c075ac0 0x7f6d3c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d41800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c076f30 0x7f6d3c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d41800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c10a910 0x7f6d3c19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d41800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c110830 con 0x7f6d3c075ac0 2024-09-11T00:29:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d41800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1106b0 con 0x7f6d3c076f30 2024-09-11T00:29:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d41800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1109b0 con 0x7f6d3c10a910 2024-09-11T00:29:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d3a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c076f30 0x7f6d3c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d3a000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c076f30 0x7f6d3c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:55592/0 (socket says 172.21.15.103:55592) 2024-09-11T00:29:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d3a000640 1 -- 172.21.15.103:0/979081926 learned_addr learned my addr 172.21.15.103:0/979081926 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.538+0000 7f6d3b400640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c10a910 0x7f6d3c19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.539+0000 7f6d3a000640 1 -- 172.21.15.103:0/979081926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c10a910 msgr2=0x7f6d3c19c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.539+0000 7f6d3a000640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c10a910 0x7f6d3c19c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.539+0000 7f6d3a000640 1 -- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c075ac0 msgr2=0x7f6d3c19af80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T00:29:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.540+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c075ac0 0x7f6d3c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.540+0000 7f6d3a000640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c075ac0 0x7f6d3c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.540+0000 7f6d3a000640 1 -- 172.21.15.103:0/979081926 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c19cac0 con 0x7f6d3c076f30 2024-09-11T00:29:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.540+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c075ac0 0x7f6d3c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:29:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.541+0000 7f6d3a000640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c076f30 0x7f6d3c19b5d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d28005ab0 tx=0x7f6d28003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.541+0000 7f6d33400640 1 -- 172.21.15.103:0/979081926 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2803f070 con 0x7f6d3c076f30 2024-09-11T00:29:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.541+0000 7f6d33400640 1 -- 172.21.15.103:0/979081926 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d28031c50 con 0x7f6d3c076f30 2024-09-11T00:29:14.546 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.541+0000 7f6d33400640 1 -- 172.21.15.103:0/979081926 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d28035030 con 0x7f6d3c076f30 2024-09-11T00:29:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.541+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c19cd50 con 0x7f6d3c076f30 2024-09-11T00:29:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.542+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c1a32a0 con 0x7f6d3c076f30 2024-09-11T00:29:14.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.545+0000 7f6d33400640 1 -- 172.21.15.103:0/979081926 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6d2803b4f0 con 0x7f6d3c076f30 2024-09-11T00:29:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.546+0000 7f6d44657740 1 Processor -- start 2024-09-11T00:29:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d44657740 1 -- start start 2024-09-11T00:29:14.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d44657740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d0689790 0x55f0d068bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d44657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d068c0c0 0x55f0d068e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d33400640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d200b5360 0x7f6d200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d44657740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d0690fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d44657740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f0d05fc6b0 con 0x55f0d0689790 2024-09-11T00:29:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d44657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0d05ebae0 con 0x55f0d068c0c0 2024-09-11T00:29:14.551 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d44657740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f0d05ef8f0 con 0x55f0d068eb00 2024-09-11T00:29:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.547+0000 7f6d33400640 1 -- 172.21.15.103:0/979081926 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6d28106ae0 con 0x7f6d3c076f30 2024-09-11T00:29:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.548+0000 7f6d3b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d0690fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.548+0000 7f6d3b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d0690fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:58992/0 (socket says 172.21.15.103:58992) 2024-09-11T00:29:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.548+0000 7f6d3b400640 1 -- 172.21.15.103:0/1058559264 learned_addr learned my addr 172.21.15.103:0/1058559264 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.548+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d068c0c0 0x55f0d068e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.548+0000 7f6d3b400640 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d0689790 0x55f0d068bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.548+0000 7f6d3a000640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d200b5360 0x7f6d200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.549+0000 7f6d3aa00640 1 -- 172.21.15.103:0/1058559264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 msgr2=0x55f0d0690fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.549+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d0690fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.549+0000 7f6d3aa00640 1 -- 172.21.15.103:0/1058559264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d0689790 msgr2=0x55f0d068bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.549+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d0689790 0x55f0d068bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.549+0000 7f6d3aa00640 1 -- 172.21.15.103:0/1058559264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0d05b8780 con 0x55f0d068c0c0 2024-09-11T00:29:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.549+0000 7f6d3a000640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d200b5360 0x7f6d200b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6d281041f0 tx=0x7f6d28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.549+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d068c0c0 0x55f0d068e4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009c40 tx=0x7f6d2c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4cd82f9ed7090ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.550+0000 7f6d27e00640 1 -- 172.21.15.103:0/1058559264 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c004030 con 0x55f0d068c0c0 2024-09-11T00:29:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.550+0000 7f6d27e00640 1 -- 172.21.15.103:0/1058559264 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d2c0035f0 con 0x55f0d068c0c0 2024-09-11T00:29:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.550+0000 7f6d27e00640 1 -- 172.21.15.103:0/1058559264 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0126b0 con 0x55f0d068c0c0 2024-09-11T00:29:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.551+0000 7f6d44657740 1 -- 172.21.15.103:0/1058559264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d068c0c0 msgr2=0x55f0d068e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.551+0000 7f6d44657740 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d068c0c0 0x55f0d068e4b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009c40 tx=0x7f6d2c010740 comp rx=0 tx=0).stop 2024-09-11T00:29:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.552+0000 7f6d44657740 1 -- 172.21.15.103:0/1058559264 shutdown_connections 2024-09-11T00:29:14.560 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.552+0000 7f6d44657740 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d0690fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.552+0000 7f6d44657740 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d068c0c0 0x55f0d068e4b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.552+0000 7f6d44657740 1 --2- 172.21.15.103:0/1058559264 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d0689790 0x55f0d068bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.552+0000 7f6d44657740 1 -- 172.21.15.103:0/1058559264 >> 172.21.15.103:0/1058559264 conn(0x55f0d06848e0 msgr2=0x55f0d0685670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.552+0000 7f6d44657740 1 -- 172.21.15.103:0/1058559264 shutdown_connections 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.552+0000 7f6d44657740 1 -- 172.21.15.103:0/1058559264 wait complete. 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 -- start start 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 Processor -- start 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0692f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d06935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0694190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f0d0523ab0 con 0x55f0d068eb00 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f0d0592570 con 0x55f0d0689790 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f0d0691ae0 con 0x55f0d068c0c0 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.553+0000 7f6d44657740 10 client.0 _ll_register_callbacks cb 0x55f0d05a1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d06935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d06935c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59000/0 (socket says 172.21.15.103:59000) 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2584765661 learned_addr learned my addr 172.21.15.103:0/2584765661 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0692f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0694190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 msgr2=0x55f0d0692f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0692f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2584765661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 msgr2=0x55f0d0694190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0694190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2584765661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0d034d790 con 0x55f0d068c0c0 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.554+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0694190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d06935c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c000c00 tx=0x7f6d2c009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59767fce7fa8ff8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d24c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d24c00640 1 -- 172.21.15.103:0/2584765661 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c02a060 con 0x55f0d068c0c0 2024-09-11T00:29:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d24c00640 1 -- 172.21.15.103:0/2584765661 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6d2c004210 con 0x55f0d068c0c0 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d2c0035f0 con 0x55f0d068c0c0 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d24c00640 1 -- 172.21.15.103:0/2584765661 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6d2c031050 con 0x55f0d068c0c0 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d44657740 10 client.26071 fetch_fsmap learned FSMap version 41 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.555+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f0d03c6620 con 0x55f0d068c0c0 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d24c00640 1 -- 172.21.15.103:0/2584765661 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d2c0035f0 con 0x55f0d068c0c0 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d24c00640 1 -- 172.21.15.103:0/2584765661 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f6d2c00f040 con 0x55f0d068c0c0 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d44657740 10 client.26071 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d44657740 10 client.26071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d44657740 20 client.26071 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d44657740 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d069e5e0 0x55f0d06a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d44657740 4 client.26071 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0d05fe600 con 0x55f0d069e5e0 2024-09-11T00:29:14.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.556+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d069e5e0 0x55f0d06a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.557+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d069e5e0 0x55f0d06a09f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455278aa3d972cd8 server_cookie=9a86e344314c295a in_seq=0 out_seq=0 2024-09-11T00:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.557+0000 7f6d24c00640 10 client.26071 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d24c00640 1 -- 172.21.15.103:0/2584765661 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0d05fe600 con 0x55f0d069e5e0 2024-09-11T00:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d24c00640 10 client.26071 handle_command_reply: tid=0 2024-09-11T00:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 client.26071 shutdown 2024-09-11T00:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 msgr2=0x55f0d06935c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d06935c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c000c00 tx=0x7f6d2c009980 comp rx=0 tx=0).stop 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 shutdown_connections 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0694190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d069e5e0 0x55f0d06a09f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d06935c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 --2- 172.21.15.103:0/2584765661 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0692f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 >> 172.21.15.103:0/2584765661 conn(0x55f0d06848e0 msgr2=0x55f0d0683a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d24c00640 0 client.26071 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 shutdown_connections 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.558+0000 7f6d44657740 1 -- 172.21.15.103:0/2584765661 wait complete. 2024-09-11T00:29:14.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.559+0000 7f6d44657740 20 client.26071 trim_cache size 0 max 16384 2024-09-11T00:29:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.588+0000 7f6d44657740 1 Processor -- start 2024-09-11T00:29:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.588+0000 7f6d44657740 1 -- start start 2024-09-11T00:29:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d44657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0685300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d44657740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068c0c0 0x55f0d0685840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d44657740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d068eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d44657740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1b0220 con 0x55f0d068c0c0 2024-09-11T00:29:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d44657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1ada90 con 0x55f0d0689790 2024-09-11T00:29:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d44657740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1ab8e0 con 0x55f0d068eb00 2024-09-11T00:29:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068c0c0 0x55f0d0685840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068c0c0 0x55f0d0685840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:59924/0 (socket says 172.21.15.103:59924) 2024-09-11T00:29:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d3aa00640 1 -- 172.21.15.103:0/3890697208 learned_addr learned my addr 172.21.15.103:0/3890697208 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d3b400640 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d068eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.589+0000 7f6d3b400640 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0685300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d3aa00640 1 -- 172.21.15.103:0/3890697208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 msgr2=0x55f0d068eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d068eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d3aa00640 1 -- 172.21.15.103:0/3890697208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 msgr2=0x55f0d0685300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0685300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d3aa00640 1 -- 172.21.15.103:0/3890697208 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0d068bef0 con 0x55f0d068c0c0 2024-09-11T00:29:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068c0c0 0x55f0d0685840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c001eb0 tx=0x7f6d2c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2354815ef4a1aa00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d27400640 1 -- 172.21.15.103:0/3890697208 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c02a070 con 0x55f0d068c0c0 2024-09-11T00:29:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d27400640 1 -- 172.21.15.103:0/3890697208 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d2c01add0 con 0x55f0d068c0c0 2024-09-11T00:29:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.590+0000 7f6d27400640 1 -- 172.21.15.103:0/3890697208 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0054f0 con 0x55f0d068c0c0 2024-09-11T00:29:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.591+0000 7f6d44657740 1 -- 172.21.15.103:0/3890697208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068c0c0 msgr2=0x55f0d0685840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.591+0000 7f6d44657740 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068c0c0 0x55f0d0685840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c001eb0 tx=0x7f6d2c005850 comp rx=0 tx=0).stop 2024-09-11T00:29:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.591+0000 7f6d44657740 1 -- 172.21.15.103:0/3890697208 shutdown_connections 2024-09-11T00:29:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.591+0000 7f6d44657740 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068eb00 0x55f0d068eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.591+0000 7f6d44657740 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068c0c0 0x55f0d0685840 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.592+0000 7f6d44657740 1 --2- 172.21.15.103:0/3890697208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0685300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.592+0000 7f6d44657740 1 -- 172.21.15.103:0/3890697208 >> 172.21.15.103:0/3890697208 conn(0x55f0d06848e0 msgr2=0x55f0d06920c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.592+0000 7f6d44657740 1 -- 172.21.15.103:0/3890697208 shutdown_connections 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.592+0000 7f6d44657740 1 -- 172.21.15.103:0/3890697208 wait complete. 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.592+0000 7f6d44657740 1 -- start start 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.592+0000 7f6d44657740 1 Processor -- start 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d44657740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0697db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d44657740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d0698450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d44657740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0699020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d44657740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1abf50 con 0x55f0d068eb00 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d44657740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1ad3c0 con 0x55f0d0689790 2024-09-11T00:29:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d44657740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c1ac620 con 0x55f0d068c0c0 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d44657740 10 client.0 _ll_register_callbacks cb 0x7f6d34009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d0698450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d3aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d0698450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59014/0 (socket says 172.21.15.103:59014) 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.593+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2844279938 learned_addr learned my addr 172.21.15.103:0/2844279938 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0697db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 msgr2=0x55f0d0697db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0697db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2844279938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 msgr2=0x55f0d0699020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0699020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3aa00640 1 -- 172.21.15.103:0/2844279938 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0d0699850 con 0x55f0d068c0c0 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0697db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d3aa00640 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d0698450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c005dc0 tx=0x7f6d2c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e02d339f6e7f010b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d24c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:29:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d24c00640 1 -- 172.21.15.103:0/2844279938 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c02a070 con 0x55f0d068c0c0 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d2c01add0 con 0x55f0d068c0c0 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.594+0000 7f6d24c00640 1 -- 172.21.15.103:0/2844279938 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6d2c03a040 con 0x55f0d068c0c0 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d24c00640 1 -- 172.21.15.103:0/2844279938 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6d2c049050 con 0x55f0d068c0c0 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 10 client.26077 fetch_fsmap learned FSMap version 41 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f0d069abc0 con 0x55f0d068c0c0 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d24c00640 1 -- 172.21.15.103:0/2844279938 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d2c01add0 con 0x55f0d068c0c0 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d24c00640 1 -- 172.21.15.103:0/2844279938 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f6d2c005490 con 0x55f0d068c0c0 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 10 client.26077 fetch_fsmap finished waiting for FSMap version 41 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 10 client.26077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 20 client.26077 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d070b840 0x55f0d070dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 4 client.26077 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0d05fa0b0 con 0x55f0d070b840 2024-09-11T00:29:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.595+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d070b840 0x55f0d070dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.596+0000 7f6d3b400640 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d070b840 0x55f0d070dc30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b18463f20f7ded6 server_cookie=8a45effec208f471 in_seq=0 out_seq=0 2024-09-11T00:29:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.596+0000 7f6d24c00640 10 client.26077 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.608+0000 7f6d24c00640 1 -- 172.21.15.103:0/2844279938 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0d05fa0b0 con 0x55f0d070b840 2024-09-11T00:29:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.608+0000 7f6d24c00640 10 client.26077 handle_command_reply: tid=0 2024-09-11T00:29:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.608+0000 7f6d44657740 1 client.26077 shutdown 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 msgr2=0x55f0d0698450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d0698450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c005dc0 tx=0x7f6d2c005f10 comp rx=0 tx=0).stop 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 shutdown_connections 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f0d068eb00 0x55f0d0699020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f0d070b840 0x55f0d070dc30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f0d068c0c0 0x55f0d0698450 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 --2- 172.21.15.103:0/2844279938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f0d0689790 0x55f0d0697db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d24c00640 0 client.26077 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 >> 172.21.15.103:0/2844279938 conn(0x55f0d06848e0 msgr2=0x55f0d068eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 shutdown_connections 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.609+0000 7f6d44657740 1 -- 172.21.15.103:0/2844279938 wait complete. 2024-09-11T00:29:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d44657740 20 client.26077 trim_cache size 0 max 16384 2024-09-11T00:29:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:29:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:29:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:29:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:29:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d200b5360 msgr2=0x7f6d200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d200b5360 0x7f6d200b7820 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6d281041f0 tx=0x7f6d28009260 comp rx=0 tx=0).stop 2024-09-11T00:29:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c076f30 msgr2=0x7f6d3c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c076f30 0x7f6d3c19b5d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d28005ab0 tx=0x7f6d28003b90 comp rx=0 tx=0).stop 2024-09-11T00:29:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 shutdown_connections 2024-09-11T00:29:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6d200b5360 0x7f6d200b7820 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d3c10a910 0x7f6d3c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.610+0000 7f6d41800640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6d3c076f30 0x7f6d3c19b5d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.611+0000 7f6d41800640 1 --2- 172.21.15.103:0/979081926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d3c075ac0 0x7f6d3c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.611+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 >> 172.21.15.103:0/979081926 conn(0x7f6d3c0fe250 msgr2=0x7f6d3c1068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.611+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 shutdown_connections 2024-09-11T00:29:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:14.611+0000 7f6d41800640 1 -- 172.21.15.103:0/979081926 wait complete. 2024-09-11T00:29:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:15 smithi183 ceph-mon[35934]: pgmap v585: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 458 op/s 2024-09-11T00:29:16.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:15 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:29:16.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:15 smithi103 ceph-mon[30578]: pgmap v585: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 458 op/s 2024-09-11T00:29:16.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:15 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:29:16.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:15 smithi117 ceph-mon[35525]: pgmap v585: 129 pgs: 129 active+clean; 69 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.4 MiB/s wr, 458 op/s 2024-09-11T00:29:16.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:15 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:29:16.300 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x9ffb01 2024-09-11T00:29:18.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:17 smithi183 ceph-mon[35934]: pgmap v586: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 371 op/s 2024-09-11T00:29:18.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:17 smithi103 ceph-mon[30578]: pgmap v586: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 371 op/s 2024-09-11T00:29:18.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:17 smithi117 ceph-mon[35525]: pgmap v586: 129 pgs: 129 active+clean; 66 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.1 MiB/s wr, 371 op/s 2024-09-11T00:29:20.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:19 smithi183 ceph-mon[35934]: pgmap v587: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2024-09-11T00:29:20.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:19 smithi103 ceph-mon[30578]: pgmap v587: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2024-09-11T00:29:20.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:19 smithi117 ceph-mon[35525]: pgmap v587: 129 pgs: 129 active+clean; 59 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 2.3 MiB/s wr, 396 op/s 2024-09-11T00:29:22.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:21 smithi183 ceph-mon[35934]: pgmap v588: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.2 MiB/s wr, 396 op/s 2024-09-11T00:29:22.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:21 smithi103 ceph-mon[30578]: pgmap v588: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.2 MiB/s wr, 396 op/s 2024-09-11T00:29:22.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:21 smithi117 ceph-mon[35525]: pgmap v588: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.2 MiB/s wr, 396 op/s 2024-09-11T00:29:22.874 INFO:tasks.workunit.client.0.smithi103.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-09-11T00:29:24.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:23 smithi183 ceph-mon[35934]: pgmap v589: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.8 MiB/s wr, 332 op/s 2024-09-11T00:29:24.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:23 smithi103 ceph-mon[30578]: pgmap v589: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.8 MiB/s wr, 332 op/s 2024-09-11T00:29:24.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:23 smithi117 ceph-mon[35525]: pgmap v589: 129 pgs: 129 active+clean; 61 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 1.8 MiB/s wr, 332 op/s 2024-09-11T00:29:26.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:25 smithi183 ceph-mon[35934]: pgmap v590: 129 pgs: 129 active+clean; 62 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 455 op/s 2024-09-11T00:29:26.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:25 smithi103 ceph-mon[30578]: pgmap v590: 129 pgs: 129 active+clean; 62 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 455 op/s 2024-09-11T00:29:26.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:25 smithi117 ceph-mon[35525]: pgmap v590: 129 pgs: 129 active+clean; 62 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.4 MiB/s wr, 455 op/s 2024-09-11T00:29:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.747+0000 7f5da1a00640 1 -- 172.21.15.103:0/1598751207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 msgr2=0x7f5d9c106740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.747+0000 7f5da1a00640 1 --2- 172.21.15.103:0/1598751207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 0x7f5d9c106740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009ca0 tx=0x7f5d9402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.748+0000 7f5da1a00640 1 -- 172.21.15.103:0/1598751207 shutdown_connections 2024-09-11T00:29:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.748+0000 7f5da1a00640 1 --2- 172.21.15.103:0/1598751207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 0x7f5d9c106740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.748+0000 7f5da1a00640 1 --2- 172.21.15.103:0/1598751207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c105b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.748+0000 7f5da1a00640 1 --2- 172.21.15.103:0/1598751207 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c06b470 0x7f5d9c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.748+0000 7f5da1a00640 1 -- 172.21.15.103:0/1598751207 >> 172.21.15.103:0/1598751207 conn(0x7f5d9c0fcf30 msgr2=0x7f5d9c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.748+0000 7f5da1a00640 1 -- 172.21.15.103:0/1598751207 shutdown_connections 2024-09-11T00:29:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.748+0000 7f5da1a00640 1 -- 172.21.15.103:0/1598751207 wait complete. 2024-09-11T00:29:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 Processor -- start 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 -- start start 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c06b470 0x7f5d9c19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 0x7f5d9c1a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c114a20 con 0x7f5d9c06b470 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1148a0 con 0x7f5d9c106290 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c114ba0 con 0x7f5d9c1036e0 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5d9b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c06b470 0x7f5d9c19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.749+0000 7f5da1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 0x7f5d9c1a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c19f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:43246/0 (socket says 172.21.15.103:43246) 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5da1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 0x7f5d9c1a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48608/0 (socket says 172.21.15.103:48608) 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 -- 172.21.15.103:0/636640324 learned_addr learned my addr 172.21.15.103:0/636640324 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 -- 172.21.15.103:0/636640324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 msgr2=0x7f5d9c1a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 0x7f5d9c1a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 -- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c06b470 msgr2=0x7f5d9c19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c06b470 0x7f5d9c19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 -- 172.21.15.103:0/636640324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c1a0c50 con 0x7f5d9c1036e0 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9be00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c06b470 0x7f5d9c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:27.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.750+0000 7f5d9b400640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c19f760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e9f0 tx=0x7f5d8800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.752+0000 7f5d98c00640 1 -- 172.21.15.103:0/636640324 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800cd80 con 0x7f5d9c1036e0 2024-09-11T00:29:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.752+0000 7f5d98c00640 1 -- 172.21.15.103:0/636640324 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d880108e0 con 0x7f5d9c1036e0 2024-09-11T00:29:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.752+0000 7f5d98c00640 1 -- 172.21.15.103:0/636640324 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88018980 con 0x7f5d9c1036e0 2024-09-11T00:29:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.752+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c1a0f40 con 0x7f5d9c1036e0 2024-09-11T00:29:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.752+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1a7610 con 0x7f5d9c1036e0 2024-09-11T00:29:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 Processor -- start 2024-09-11T00:29:27.756 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 -- start start 2024-09-11T00:29:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd59dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd50e6b0 con 0x55dcdd59b970 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd4fdae0 con 0x55dcdd59e2a0 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.755+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd5018f0 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5da1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd59dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48612/0 (socket says 172.21.15.103:48612) 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 -- 172.21.15.103:0/3509922361 learned_addr learned my addr 172.21.15.103:0/3509922361 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5da1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd59dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41148/0 (socket says 172.21.15.103:41148) 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d98c00640 1 -- 172.21.15.103:0/636640324 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5d88010430 con 0x7f5d9c1036e0 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 -- 172.21.15.103:0/3509922361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd5a0ce0 msgr2=0x55dcdd5a31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 --2- 172.21.15.103:0/3509922361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 -- 172.21.15.103:0/3509922361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 msgr2=0x55dcdd59dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 --2- 172.21.15.103:0/3509922361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd59dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.756+0000 7f5d9be00640 1 -- 172.21.15.103:0/3509922361 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdd4ca780 con 0x55dcdd59e2a0 2024-09-11T00:29:27.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.757+0000 7f5d9be00640 1 --2- 172.21.15.103:0/3509922361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a0690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0115f0 tx=0x7f5d8c011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac0df090de2c2c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.757+0000 7f5d91600640 1 -- 172.21.15.103:0/3509922361 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c01e070 con 0x55dcdd59e2a0 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.757+0000 7f5d91600640 1 -- 172.21.15.103:0/3509922361 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c00faf0 con 0x55dcdd59e2a0 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.757+0000 7f5d91600640 1 -- 172.21.15.103:0/3509922361 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c019bc0 con 0x55dcdd59e2a0 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 -- 172.21.15.103:0/3509922361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 msgr2=0x55dcdd5a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/3509922361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a0690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0115f0 tx=0x7f5d8c011ab0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5d98c00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d800b5360 0x7f5d800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 -- 172.21.15.103:0/3509922361 shutdown_connections 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/3509922361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/3509922361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a0690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/3509922361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd59dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 -- 172.21.15.103:0/3509922361 >> 172.21.15.103:0/3509922361 conn(0x55dcdd596ac0 msgr2=0x55dcdd597850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 -- 172.21.15.103:0/3509922361 shutdown_connections 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5d98c00640 1 -- 172.21.15.103:0/636640324 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f5d880e61c0 con 0x7f5d9c1036e0 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 -- 172.21.15.103:0/3509922361 wait complete. 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da1000640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d800b5360 0x7f5d800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.758+0000 7f5da5fd2740 1 -- start start 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 1 Processor -- start 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da1000640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d800b5360 0x7f5d800b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005e30 tx=0x7f5d94031af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59b970 0x55dcdd5a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd435ab0 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd4a4570 con 0x55dcdd59e2a0 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd5a3cc0 con 0x55dcdd59b970 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5da5fd2740 10 client.0 _ll_register_callbacks cb 0x55dcdd4b3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59b970 0x55dcdd5a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5d9b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59b970 0x55dcdd5a50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:43260/0 (socket says 172.21.15.103:43260) 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.759+0000 7f5d9be00640 1 -- 172.21.15.103:0/946042665 learned_addr learned my addr 172.21.15.103:0/946042665 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d9be00640 1 -- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 msgr2=0x55dcdd5a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d9be00640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d9be00640 1 -- 172.21.15.103:0/946042665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 msgr2=0x55dcdd5a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5da1000640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d9be00640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d9be00640 1 -- 172.21.15.103:0/946042665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcdd25f790 con 0x55dcdd59b970 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d9b400640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5da1000640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d9be00640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59b970 0x55dcdd5a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009560 tx=0x7f5d8c019640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=353eb7ce7c4968 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d86000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d86000640 1 -- 172.21.15.103:0/946042665 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c027060 con 0x55dcdd59b970 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5d86000640 1 -- 172.21.15.103:0/946042665 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f5d8c00f6c0 con 0x55dcdd59b970 2024-09-11T00:29:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.760+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c00faf0 con 0x55dcdd59b970 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5d86000640 1 -- 172.21.15.103:0/946042665 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c02e050 con 0x55dcdd59b970 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 10 client.26086 fetch_fsmap learned FSMap version 42 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dcdd2d8620 con 0x55dcdd59b970 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5d86000640 1 -- 172.21.15.103:0/946042665 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c00faf0 con 0x55dcdd59b970 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5d86000640 1 -- 172.21.15.103:0/946042665 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f5d8c021070 con 0x55dcdd59b970 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 10 client.26086 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 20 client.26086 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd5b0640 0x55dcdd5b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 4 client.26086 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcdd510600 con 0x55dcdd5b0640 2024-09-11T00:29:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.761+0000 7f5d9b400640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd5b0640 0x55dcdd5b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.762+0000 7f5d9b400640 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd5b0640 0x55dcdd5b2a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58c0854e2c1531b server_cookie=6c72a8668552d89d in_seq=0 out_seq=0 2024-09-11T00:29:27.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.762+0000 7f5d86000640 10 client.26086 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5d86000640 1 -- 172.21.15.103:0/946042665 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dcdd510600 con 0x55dcdd5b0640 2024-09-11T00:29:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5d86000640 10 client.26086 handle_command_reply: tid=0 2024-09-11T00:29:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 client.26086 shutdown 2024-09-11T00:29:27.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59b970 msgr2=0x55dcdd5a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59b970 0x55dcdd5a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009560 tx=0x7f5d8c019640 comp rx=0 tx=0).stop 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 shutdown_connections 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd5b0640 0x55dcdd5b2a30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/946042665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59b970 0x55dcdd5a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 >> 172.21.15.103:0/946042665 conn(0x55dcdd596ac0 msgr2=0x55dcdd595bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5d86000640 0 client.26086 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 shutdown_connections 2024-09-11T00:29:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 1 -- 172.21.15.103:0/946042665 wait complete. 2024-09-11T00:29:27.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.763+0000 7f5da5fd2740 20 client.26086 trim_cache size 0 max 16384 2024-09-11T00:29:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 Processor -- start 2024-09-11T00:29:27.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 -- start start 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59b970 0x55dcdd5b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b44f0 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b1d90 con 0x55dcdd59b970 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.783+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1afbe0 con 0x55dcdd59e2a0 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59b970 0x55dcdd5b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59b970 0x55dcdd5b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:48642/0 (socket says 172.21.15.103:48642) 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 -- 172.21.15.103:0/2393558892 learned_addr learned my addr 172.21.15.103:0/2393558892 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5da1000640 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9b400640 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 -- 172.21.15.103:0/2393558892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 msgr2=0x55dcdd5a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 -- 172.21.15.103:0/2393558892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 msgr2=0x55dcdd5a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.784+0000 7f5d9be00640 1 -- 172.21.15.103:0/2393558892 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdd5a4090 con 0x55dcdd59b970 2024-09-11T00:29:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.785+0000 7f5d9be00640 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59b970 0x55dcdd5b2650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=195d5851992d97ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.785+0000 7f5d90c00640 1 -- 172.21.15.103:0/2393558892 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c015070 con 0x55dcdd59b970 2024-09-11T00:29:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.785+0000 7f5d90c00640 1 -- 172.21.15.103:0/2393558892 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c004d30 con 0x55dcdd59b970 2024-09-11T00:29:27.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.785+0000 7f5d90c00640 1 -- 172.21.15.103:0/2393558892 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c027070 con 0x55dcdd59b970 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.786+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2393558892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59b970 msgr2=0x55dcdd5b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.786+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59b970 0x55dcdd5b2650 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000c00 tx=0x7f5d8c005970 comp rx=0 tx=0).stop 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.786+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2393558892 shutdown_connections 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.786+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.786+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.786+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2393558892 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd59b970 0x55dcdd5b2650 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.786+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2393558892 >> 172.21.15.103:0/2393558892 conn(0x55dcdd596ac0 msgr2=0x55dcdd5a42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.787+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2393558892 shutdown_connections 2024-09-11T00:29:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.787+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2393558892 wait complete. 2024-09-11T00:29:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.787+0000 7f5da5fd2740 1 -- start start 2024-09-11T00:29:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.787+0000 7f5da5fd2740 1 Processor -- start 2024-09-11T00:29:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd5ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.790 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5ab6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da5fd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5abc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b0250 con 0x55dcdd59b970 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b16c0 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c1b0920 con 0x55dcdd59e2a0 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da5fd2740 10 client.0 _ll_register_callbacks cb 0x7f5d8c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5d9b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5ab6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5d9b400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5ab6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:43276/0 (socket says 172.21.15.103:43276) 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5d9b400640 1 -- 172.21.15.103:0/2602824935 learned_addr learned my addr 172.21.15.103:0/2602824935 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5da1000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5abc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.788+0000 7f5d9be00640 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd5ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5da1000640 1 -- 172.21.15.103:0/2602824935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 msgr2=0x55dcdd5ab6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5da1000640 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5ab6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5da1000640 1 -- 172.21.15.103:0/2602824935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 msgr2=0x55dcdd5ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5da1000640 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd5ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5da1000640 1 -- 172.21.15.103:0/2602824935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcdd5ac380 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5d9b400640 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5da1000640 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5abc10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d9402ade0 tx=0x7f5d9402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c030ed38178b8fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.789+0000 7f5d86000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5d86000640 1 -- 172.21.15.103:0/2602824935 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94058060 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c004d30 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5d86000640 1 -- 172.21.15.103:0/2602824935 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f5d94004250 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5d86000640 1 -- 172.21.15.103:0/2602824935 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5d9405f050 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5da5fd2740 10 client.26072 fetch_fsmap learned FSMap version 42 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dcdd61d550 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5d86000640 1 -- 172.21.15.103:0/2602824935 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c004d30 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5d86000640 1 -- 172.21.15.103:0/2602824935 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f5d94052070 con 0x55dcdd5a0ce0 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5da5fd2740 10 client.26072 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5da5fd2740 10 client.26072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5da5fd2740 20 client.26072 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.790+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd61df60 0x55dcdd620380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.791+0000 7f5da5fd2740 4 client.26072 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.791+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcdd50c0b0 con 0x55dcdd61df60 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.791+0000 7f5d9be00640 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd61df60 0x55dcdd620380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:27.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.791+0000 7f5d9be00640 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd61df60 0x55dcdd620380 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98bee8a3d83d93a1 server_cookie=9f4f6bf8b1380fef in_seq=0 out_seq=0 2024-09-11T00:29:27.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.791+0000 7f5d86000640 10 client.26072 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:27.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5d86000640 1 -- 172.21.15.103:0/2602824935 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcdd50c0b0 con 0x55dcdd61df60 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5d86000640 10 client.26072 handle_command_reply: tid=0 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 client.26072 shutdown 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd5a0ce0 msgr2=0x55dcdd5abc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5abc10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5d9402ade0 tx=0x7f5d9402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 shutdown_connections 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55dcdd5a0ce0 0x55dcdd5abc10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55dcdd61df60 0x55dcdd620380 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dcdd59e2a0 0x55dcdd5ab6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 --2- 172.21.15.103:0/2602824935 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dcdd59b970 0x55dcdd5ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5d86000640 0 client.26072 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:27.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 >> 172.21.15.103:0/2602824935 conn(0x55dcdd596ac0 msgr2=0x55dcdd597320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.803+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 shutdown_connections 2024-09-11T00:29:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.804+0000 7f5da5fd2740 1 -- 172.21.15.103:0/2602824935 wait complete. 2024-09-11T00:29:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.804+0000 7f5da5fd2740 20 client.26072 trim_cache size 0 max 16384 2024-09-11T00:29:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:29:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:29:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:29:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:29:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d800b5360 msgr2=0x7f5d800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d800b5360 0x7f5d800b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5d94005e30 tx=0x7f5d94031af0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 msgr2=0x7f5d9c19f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c19f760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e9f0 tx=0x7f5d8800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 shutdown_connections 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5d800b5360 0x7f5d800b7820 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d9c106290 0x7f5d9c1a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d9c1036e0 0x7f5d9c19f760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 --2- 172.21.15.103:0/636640324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5d9c06b470 0x7f5d9c19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 >> 172.21.15.103:0/636640324 conn(0x7f5d9c0fcf30 msgr2=0x7f5d9c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 shutdown_connections 2024-09-11T00:29:27.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:27.805+0000 7f5da1a00640 1 -- 172.21.15.103:0/636640324 wait complete. 2024-09-11T00:29:28.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:27 smithi183 ceph-mon[35934]: pgmap v591: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 368 op/s 2024-09-11T00:29:28.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:27 smithi103 ceph-mon[30578]: pgmap v591: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 368 op/s 2024-09-11T00:29:28.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:27 smithi117 ceph-mon[35525]: pgmap v591: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.1 MiB/s wr, 368 op/s 2024-09-11T00:29:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:29 smithi183 ceph-mon[35934]: pgmap v592: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 392 op/s 2024-09-11T00:29:30.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:29 smithi103 ceph-mon[30578]: pgmap v592: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 392 op/s 2024-09-11T00:29:30.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:29 smithi117 ceph-mon[35525]: pgmap v592: 129 pgs: 129 active+clean; 60 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.2 MiB/s wr, 392 op/s 2024-09-11T00:29:32.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:31 smithi183 ceph-mon[35934]: pgmap v593: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 407 op/s 2024-09-11T00:29:32.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:31 smithi103 ceph-mon[30578]: pgmap v593: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 407 op/s 2024-09-11T00:29:32.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:31 smithi117 ceph-mon[35525]: pgmap v593: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.2 MiB/s wr, 407 op/s 2024-09-11T00:29:34.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:33 smithi183 ceph-mon[35934]: pgmap v594: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-09-11T00:29:34.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:33 smithi103 ceph-mon[30578]: pgmap v594: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-09-11T00:29:34.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:33 smithi117 ceph-mon[35525]: pgmap v594: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.8 MiB/s wr, 350 op/s 2024-09-11T00:29:36.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:35 smithi183 ceph-mon[35934]: pgmap v595: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.3 MiB/s wr, 415 op/s 2024-09-11T00:29:36.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:35 smithi103 ceph-mon[30578]: pgmap v595: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.3 MiB/s wr, 415 op/s 2024-09-11T00:29:36.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:35 smithi117 ceph-mon[35525]: pgmap v595: 129 pgs: 129 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.3 MiB/s wr, 415 op/s 2024-09-11T00:29:37.822 INFO:tasks.workunit.client.0.smithi103.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-09-11T00:29:37.822 INFO:tasks.workunit.client.0.smithi103.stdout:All 50000 operations completed A-OK! 2024-09-11T00:29:37.826 INFO:tasks.workunit.client.0.smithi103.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-09-11T00:29:37.833 INFO:tasks.workunit.client.0.smithi103.stdout:Seed set to 1 2024-09-11T00:29:38.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:37 smithi183 ceph-mon[35934]: pgmap v596: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-09-11T00:29:38.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:37 smithi103 ceph-mon[30578]: pgmap v596: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-09-11T00:29:38.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:37 smithi117 ceph-mon[35525]: pgmap v596: 129 pgs: 129 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.0 MiB/s wr, 335 op/s 2024-09-11T00:29:39.102 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-11T00:29:39.103 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-11T00:29:39.105 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-11T00:29:39.105 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-11T00:29:39.105 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support clone range, disabling! 2024-09-11T00:29:39.109 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-11T00:29:39.111 INFO:tasks.workunit.client.0.smithi103.stderr:main: filesystem does not support exchange range, disabling! 2024-09-11T00:29:39.111 INFO:tasks.workunit.client.0.smithi103.stdout:skipping zero size read 2024-09-11T00:29:39.128 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x1c37b90 2024-09-11T00:29:39.135 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x5bd330d 2024-09-11T00:29:39.164 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0x637bf99 2024-09-11T00:29:39.564 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0x63d9b9c 2024-09-11T00:29:40.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:40 smithi183 ceph-mon[35934]: pgmap v597: 129 pgs: 129 active+clean; 67 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.2 MiB/s wr, 390 op/s 2024-09-11T00:29:40.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:40 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:29:40.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:40 smithi103 ceph-mon[30578]: pgmap v597: 129 pgs: 129 active+clean; 67 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.2 MiB/s wr, 390 op/s 2024-09-11T00:29:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:40 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:29:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:40 smithi117 ceph-mon[35525]: pgmap v597: 129 pgs: 129 active+clean; 67 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.2 MiB/s wr, 390 op/s 2024-09-11T00:29:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:40 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.939+0000 7f70ab200640 1 -- 172.21.15.103:0/3896323468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 msgr2=0x7f70a4104fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.939+0000 7f70ab200640 1 --2- 172.21.15.103:0/3896323468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a4104fa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7094009930 tx=0x7f709402a440 comp rx=0 tx=0).stop 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.939+0000 7f70ab200640 1 -- 172.21.15.103:0/3896323468 shutdown_connections 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.939+0000 7f70ab200640 1 --2- 172.21.15.103:0/3896323468 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a41066b0 0x7f70a410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.939+0000 7f70ab200640 1 --2- 172.21.15.103:0/3896323468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a4105cf0 0x7f70a4106170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.939+0000 7f70ab200640 1 --2- 172.21.15.103:0/3896323468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a4104fa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.939+0000 7f70ab200640 1 -- 172.21.15.103:0/3896323468 >> 172.21.15.103:0/3896323468 conn(0x7f70a4100460 msgr2=0x7f70a4102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.940+0000 7f70ab200640 1 -- 172.21.15.103:0/3896323468 shutdown_connections 2024-09-11T00:29:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.940+0000 7f70ab200640 1 -- 172.21.15.103:0/3896323468 wait complete. 2024-09-11T00:29:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.940+0000 7f70ab200640 1 Processor -- start 2024-09-11T00:29:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70ab200640 1 -- start start 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70ab200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a419afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70ab200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a4105cf0 0x7f70a419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70ab200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a41066b0 0x7f70a419c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70ab200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4110860 con 0x7f70a41066b0 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70ab200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41106e0 con 0x7f70a4105cf0 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70ab200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41109e0 con 0x7f70a4104ba0 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a419afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a419afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:45062/0 (socket says 172.21.15.103:45062) 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70a3e00640 1 -- 172.21.15.103:0/268375286 learned_addr learned my addr 172.21.15.103:0/268375286 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f709b400640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a4105cf0 0x7f70a419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.941+0000 7f70a9200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a41066b0 0x7f70a419c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a3e00640 1 -- 172.21.15.103:0/268375286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a4105cf0 msgr2=0x7f70a419b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a3e00640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a4105cf0 0x7f70a419b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a3e00640 1 -- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a41066b0 msgr2=0x7f70a419c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a3e00640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a41066b0 0x7f70a419c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a3e00640 1 -- 172.21.15.103:0/268375286 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a419caf0 con 0x7f70a4104ba0 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a9200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a41066b0 0x7f70a419c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a3e00640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a419afb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7094031500 tx=0x7f7094003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a1600640 1 -- 172.21.15.103:0/268375286 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709403f070 con 0x7f70a4104ba0 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a1600640 1 -- 172.21.15.103:0/268375286 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7094031c50 con 0x7f70a4104ba0 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a419cd80 con 0x7f70a4104ba0 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.942+0000 7f70a1600640 1 -- 172.21.15.103:0/268375286 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094035030 con 0x7f70a4104ba0 2024-09-11T00:29:40.946 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.943+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a41a34b0 con 0x7f70a4104ba0 2024-09-11T00:29:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.945+0000 7f70ae027740 1 Processor -- start 2024-09-11T00:29:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70ae027740 1 -- start start 2024-09-11T00:29:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70ae027740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa3790 0x563d4eaa5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70ae027740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaa84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70ae027740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4eaaafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70ae027740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d4ea166b0 con 0x563d4eaa60c0 2024-09-11T00:29:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70ae027740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d4ea05ae0 con 0x563d4eaa8b00 2024-09-11T00:29:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70ae027740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d4ea098f0 con 0x563d4eaa3790 2024-09-11T00:29:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70a1600640 1 -- 172.21.15.103:0/268375286 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f709403b460 con 0x7f70a4104ba0 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70a9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa3790 0x563d4eaa5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70a9200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa3790 0x563d4eaa5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:45066/0 (socket says 172.21.15.103:45066) 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70a9200640 1 -- 172.21.15.103:0/3404512107 learned_addr learned my addr 172.21.15.103:0/3404512107 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f709b400640 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4eaaafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.946+0000 7f70a3e00640 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaa84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f70a9200640 1 -- 172.21.15.103:0/3404512107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 msgr2=0x563d4eaaafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f70a9200640 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4eaaafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f70a9200640 1 -- 172.21.15.103:0/3404512107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 msgr2=0x563d4eaa84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f70a9200640 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaa84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f70a9200640 1 -- 172.21.15.103:0/3404512107 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4e9d2780 con 0x563d4eaa3790 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f709b400640 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4eaaafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f70a9200640 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa3790 0x563d4eaa5b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f709c010480 tx=0x7f709c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316681c5bc0a8b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f7099600640 1 -- 172.21.15.103:0/3404512107 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c011760 con 0x563d4eaa3790 2024-09-11T00:29:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f7099600640 1 -- 172.21.15.103:0/3404512107 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c011db0 con 0x563d4eaa3790 2024-09-11T00:29:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.947+0000 7f7099600640 1 -- 172.21.15.103:0/3404512107 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c01a380 con 0x563d4eaa3790 2024-09-11T00:29:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 -- 172.21.15.103:0/3404512107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa3790 msgr2=0x563d4eaa5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa3790 0x563d4eaa5b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f709c010480 tx=0x7f709c009f90 comp rx=0 tx=0).stop 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 -- 172.21.15.103:0/3404512107 shutdown_connections 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4eaaafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaa84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 --2- 172.21.15.103:0/3404512107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa3790 0x563d4eaa5b80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 -- 172.21.15.103:0/3404512107 >> 172.21.15.103:0/3404512107 conn(0x563d4ea9e8e0 msgr2=0x563d4ea9f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 -- 172.21.15.103:0/3404512107 shutdown_connections 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70ae027740 1 -- 172.21.15.103:0/3404512107 wait complete. 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.948+0000 7f70a1600640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f70880b5340 0x7f70880b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 -- start start 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70a1600640 1 -- 172.21.15.103:0/268375286 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f7094106990 con 0x7f70a4104ba0 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70a9200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f70880b5340 0x7f70880b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 Processor -- start 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eaaceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eaae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d4e93dab0 con 0x563d4eaa60c0 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x563d4e9ac570 con 0x563d4eaa3790 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.949+0000 7f70ae027740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d4eaabae0 con 0x563d4eaa8b00 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70ae027740 10 client.0 _ll_register_callbacks cb 0x563d4e9bb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a9200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f70880b5340 0x7f70880b7800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f709c010480 tx=0x7f709c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eaaceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eaaceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:55686/0 (socket says 172.21.15.103:55686) 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 -- 172.21.15.103:0/3835556203 learned_addr learned my addr 172.21.15.103:0/3835556203 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f709b400640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a9200640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eaae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 -- 172.21.15.103:0/3835556203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 msgr2=0x563d4eaae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eaae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 -- 172.21.15.103:0/3835556203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 msgr2=0x563d4eaad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 -- 172.21.15.103:0/3835556203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4e767790 con 0x563d4eaa3790 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a9200640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eaae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f70a3e00640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eaaceb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f70940cccf0 tx=0x7f709402a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284ab400dfb537eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.950+0000 7f7092000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f7092000640 1 -- 172.21.15.103:0/3835556203 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094049040 con 0x563d4eaa3790 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f7092000640 1 -- 172.21.15.103:0/3835556203 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f70940d6060 con 0x563d4eaa3790 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f709c011db0 con 0x563d4eaa3790 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f7092000640 1 -- 172.21.15.103:0/3835556203 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7094031e00 con 0x563d4eaa3790 2024-09-11T00:29:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 10 client.26084 fetch_fsmap learned FSMap version 43 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563d4e7e0620 con 0x563d4eaa3790 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f7092000640 1 -- 172.21.15.103:0/3835556203 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f709c011db0 con 0x563d4eaa3790 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f7092000640 1 -- 172.21.15.103:0/3835556203 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f70940cbfc0 con 0x563d4eaa3790 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 10 client.26084 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 20 client.26084 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eab8690 0x563d4eabaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 4 client.26084 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.951+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d4ea18600 con 0x563d4eab8690 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.952+0000 7f709b400640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eab8690 0x563d4eabaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.952+0000 7f709b400640 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eab8690 0x563d4eabaae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb34fc6cda9acae server_cookie=80bfdaafe45f54d6 in_seq=0 out_seq=0 2024-09-11T00:29:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.952+0000 7f7092000640 10 client.26084 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f7092000640 1 -- 172.21.15.103:0/3835556203 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d4ea18600 con 0x563d4eab8690 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f7092000640 10 client.26084 handle_command_reply: tid=0 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 client.26084 shutdown 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 msgr2=0x563d4eaaceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eaaceb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f70940cccf0 tx=0x7f709402a440 comp rx=0 tx=0).stop 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 shutdown_connections 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eab8690 0x563d4eabaae0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eaae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eaad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f7092000640 0 client.26084 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 --2- 172.21.15.103:0/3835556203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eaaceb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.953+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 >> 172.21.15.103:0/3835556203 conn(0x563d4ea9e8e0 msgr2=0x563d4ea9da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.954+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 shutdown_connections 2024-09-11T00:29:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.954+0000 7f70ae027740 1 -- 172.21.15.103:0/3835556203 wait complete. 2024-09-11T00:29:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.954+0000 7f70ae027740 20 client.26084 trim_cache size 0 max 16384 2024-09-11T00:29:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.969+0000 7f70ae027740 1 Processor -- start 2024-09-11T00:29:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.969+0000 7f70ae027740 1 -- start start 2024-09-11T00:29:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70ae027740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa3790 0x563d4eaa2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70ae027740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa60c0 0x563d4eaa31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70ae027740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4ea9f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70ae027740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41b00e0 con 0x563d4eaa3790 2024-09-11T00:29:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70ae027740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41ad9b0 con 0x563d4eaa8b00 2024-09-11T00:29:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70ae027740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41ab800 con 0x563d4eaa60c0 2024-09-11T00:29:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa3790 0x563d4eaa2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa3790 0x563d4eaa2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60018/0 (socket says 172.21.15.103:60018) 2024-09-11T00:29:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 -- 172.21.15.103:0/1116972164 learned_addr learned my addr 172.21.15.103:0/1116972164 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f709b400640 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa60c0 0x563d4eaa31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a9200640 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4ea9f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 -- 172.21.15.103:0/1116972164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa60c0 msgr2=0x563d4eaa31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa60c0 0x563d4eaa31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 -- 172.21.15.103:0/1116972164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 msgr2=0x563d4ea9f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4ea9f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.970+0000 7f70a3e00640 1 -- 172.21.15.103:0/1116972164 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4eaa5ef0 con 0x563d4eaa3790 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.971+0000 7f70a3e00640 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa3790 0x563d4eaa2c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7094005ab0 tx=0x7f7094009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2796f5b7b1609973 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.971+0000 7f7098c00640 1 -- 172.21.15.103:0/1116972164 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094049070 con 0x563d4eaa3790 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.971+0000 7f7098c00640 1 -- 172.21.15.103:0/1116972164 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70940cc870 con 0x563d4eaa3790 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.971+0000 7f7098c00640 1 -- 172.21.15.103:0/1116972164 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709407c040 con 0x563d4eaa3790 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 -- 172.21.15.103:0/1116972164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa3790 msgr2=0x563d4eaa2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa3790 0x563d4eaa2c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7094005ab0 tx=0x7f7094009260 comp rx=0 tx=0).stop 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 -- 172.21.15.103:0/1116972164 shutdown_connections 2024-09-11T00:29:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa8b00 0x563d4ea9f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa60c0 0x563d4eaa31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 --2- 172.21.15.103:0/1116972164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa3790 0x563d4eaa2c80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 -- 172.21.15.103:0/1116972164 >> 172.21.15.103:0/1116972164 conn(0x563d4ea9e8e0 msgr2=0x563d4eaac090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 -- 172.21.15.103:0/1116972164 shutdown_connections 2024-09-11T00:29:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 -- 172.21.15.103:0/1116972164 wait complete. 2024-09-11T00:29:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.972+0000 7f70ae027740 1 -- start start 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 1 Processor -- start 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eab48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eab35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eab3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41abe70 con 0x563d4eaa60c0 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41ad2e0 con 0x563d4eaa3790 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f70a41ac540 con 0x563d4eaa8b00 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70ae027740 10 client.0 _ll_register_callbacks cb 0x7f7094005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f709b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eab35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f709b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eab35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60024/0 (socket says 172.21.15.103:60024) 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f709b400640 1 -- 172.21.15.103:0/962785097 learned_addr learned my addr 172.21.15.103:0/962785097 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70a9200640 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eab3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.973+0000 7f70a3e00640 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eab48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f709b400640 1 -- 172.21.15.103:0/962785097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 msgr2=0x563d4eab3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f709b400640 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eab3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f709b400640 1 -- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 msgr2=0x563d4eab48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f709b400640 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eab48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f709b400640 1 -- 172.21.15.103:0/962785097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4eab4310 con 0x563d4eaa60c0 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f709b400640 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eab35a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f708c005d00 tx=0x7f708c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08dc446546e364b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f7092000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f7092000640 1 -- 172.21.15.103:0/962785097 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c01f600 con 0x563d4eaa60c0 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f7092000640 1 -- 172.21.15.103:0/962785097 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f708c01f7a0 con 0x563d4eaa60c0 2024-09-11T00:29:40.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70940cc870 con 0x563d4eaa60c0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.974+0000 7f7092000640 1 -- 172.21.15.103:0/962785097 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f708c01f9a0 con 0x563d4eaa60c0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 10 client.16038 fetch_fsmap learned FSMap version 43 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563d4eb256f0 con 0x563d4eaa60c0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f7092000640 1 -- 172.21.15.103:0/962785097 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70940cc870 con 0x563d4eaa60c0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f7092000640 1 -- 172.21.15.103:0/962785097 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f708c010430 con 0x563d4eaa60c0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 10 client.16038 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 10 client.16038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 20 client.16038 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eb260d0 0x563d4eb284f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 4 client.16038 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.975+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d4ea140b0 con 0x563d4eb260d0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.976+0000 7f70a3e00640 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eb260d0 0x563d4eb284f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.976+0000 7f70a3e00640 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eb260d0 0x563d4eb284f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42d399d39910609 server_cookie=339df21abe8c850d in_seq=0 out_seq=0 2024-09-11T00:29:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.976+0000 7f7092000640 10 client.16038 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f7092000640 1 -- 172.21.15.103:0/962785097 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d4ea140b0 con 0x563d4eb260d0 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f7092000640 10 client.16038 handle_command_reply: tid=0 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f70ae027740 1 client.16038 shutdown 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 msgr2=0x563d4eab35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f70ae027740 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eab35a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f708c005d00 tx=0x7f708c0039c0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 shutdown_connections 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f70ae027740 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x563d4eb260d0 0x563d4eb284f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.997+0000 7f70ae027740 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d4eaa8b00 0x563d4eab3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ae027740 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d4eaa60c0 0x563d4eab35a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ae027740 1 --2- 172.21.15.103:0/962785097 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x563d4eaa3790 0x563d4eab48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 >> 172.21.15.103:0/962785097 conn(0x563d4ea9e8e0 msgr2=0x563d4ea9f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f7092000640 0 client.16038 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 shutdown_connections 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ae027740 1 -- 172.21.15.103:0/962785097 wait complete. 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ae027740 20 client.16038 trim_cache size 0 max 16384 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:29:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:29:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f70880b5340 msgr2=0x7f70880b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ab200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f70880b5340 0x7f70880b7800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f709c010480 tx=0x7f709c005fd0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 msgr2=0x7f70a419afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.998+0000 7f70ab200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a419afb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7094031500 tx=0x7f7094003c10 comp rx=0 tx=0).stop 2024-09-11T00:29:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 shutdown_connections 2024-09-11T00:29:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f70880b5340 0x7f70880b7800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f70a41066b0 0x7f70a419c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70a4105cf0 0x7f70a419b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 --2- 172.21.15.103:0/268375286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f70a4104ba0 0x7f70a419afb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 >> 172.21.15.103:0/268375286 conn(0x7f70a4100460 msgr2=0x7f70a4109080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 shutdown_connections 2024-09-11T00:29:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:40.999+0000 7f70ab200640 1 -- 172.21.15.103:0/268375286 wait complete. 2024-09-11T00:29:42.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:42 smithi183 ceph-mon[35934]: pgmap v598: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-09-11T00:29:42.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:42 smithi103 ceph-mon[30578]: pgmap v598: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-09-11T00:29:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:42 smithi117 ceph-mon[35525]: pgmap v598: 129 pgs: 129 active+clean; 66 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.9 MiB/s wr, 355 op/s 2024-09-11T00:29:44.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:44 smithi103 ceph-mon[30578]: pgmap v599: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-09-11T00:29:44.471 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:44 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:29:44.489 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:44 smithi117 ceph-mon[35525]: pgmap v599: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-09-11T00:29:44.489 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:44 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:29:44.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:44 smithi183 ceph-mon[35934]: pgmap v599: 129 pgs: 129 active+clean; 70 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 307 op/s 2024-09-11T00:29:44.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:44 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-11T00:29:45.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-11T00:29:45.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-11T00:29:45.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:29:45.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:29:45.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:29:45.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-11T00:29:45.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-11T00:29:45.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-11T00:29:45.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-11T00:29:45.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:29:45.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:29:45.672 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi117 to 176.0M 2024-09-11T00:29:45.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-11T00:29:45.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: Unable to set osd_memory_target on smithi117 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-11T00:29:45.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-11T00:29:45.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:29:45.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:29:45.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:29:46.532 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:46 smithi183 ceph-mon[35934]: pgmap v600: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.8 MiB/s wr, 377 op/s 2024-09-11T00:29:46.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:46 smithi103 ceph-mon[30578]: pgmap v600: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.8 MiB/s wr, 377 op/s 2024-09-11T00:29:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:46 smithi117 ceph-mon[35525]: pgmap v600: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.8 MiB/s wr, 377 op/s 2024-09-11T00:29:48.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:48 smithi183 ceph-mon[35934]: pgmap v601: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-09-11T00:29:48.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:48 smithi103 ceph-mon[30578]: pgmap v601: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-09-11T00:29:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:48 smithi117 ceph-mon[35525]: pgmap v601: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.4 MiB/s wr, 334 op/s 2024-09-11T00:29:50.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:50 smithi183 ceph-mon[35934]: pgmap v602: 129 pgs: 129 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.4 MiB/s wr, 386 op/s 2024-09-11T00:29:50.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:50 smithi103 ceph-mon[30578]: pgmap v602: 129 pgs: 129 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.4 MiB/s wr, 386 op/s 2024-09-11T00:29:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:50 smithi117 ceph-mon[35525]: pgmap v602: 129 pgs: 129 active+clean; 83 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.4 MiB/s wr, 386 op/s 2024-09-11T00:29:52.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:52 smithi183 ceph-mon[35934]: pgmap v603: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 358 op/s 2024-09-11T00:29:52.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:52 smithi103 ceph-mon[30578]: pgmap v603: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 358 op/s 2024-09-11T00:29:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:52 smithi117 ceph-mon[35525]: pgmap v603: 129 pgs: 129 active+clean; 79 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.1 MiB/s wr, 358 op/s 2024-09-11T00:29:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 -- 172.21.15.103:0/3744687901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c103ad0 msgr2=0x7fc71c103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 --2- 172.21.15.103:0/3744687901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c103ad0 0x7fc71c103f50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc708009930 tx=0x7fc70802a440 comp rx=0 tx=0).stop 2024-09-11T00:29:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 -- 172.21.15.103:0/3744687901 shutdown_connections 2024-09-11T00:29:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 --2- 172.21.15.103:0/3744687901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc71c104490 0x7fc71c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 --2- 172.21.15.103:0/3744687901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c103ad0 0x7fc71c103f50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 --2- 172.21.15.103:0/3744687901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c102980 0x7fc71c102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 -- 172.21.15.103:0/3744687901 >> 172.21.15.103:0/3744687901 conn(0x7fc71c0fe240 msgr2=0x7fc71c100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 -- 172.21.15.103:0/3744687901 shutdown_connections 2024-09-11T00:29:54.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.144+0000 7fc722400640 1 -- 172.21.15.103:0/3744687901 wait complete. 2024-09-11T00:29:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 Processor -- start 2024-09-11T00:29:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 -- start start 2024-09-11T00:29:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102980 0x7fc71c19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c103ad0 0x7fc71c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc71c104490 0x7fc71c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10e5f0 con 0x7fc71c102980 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10e470 con 0x7fc71c103ad0 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.145+0000 7fc722400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c10e770 con 0x7fc71c104490 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c103ad0 0x7fc71c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c103ad0 0x7fc71c19b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:49914/0 (socket says 172.21.15.103:49914) 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 -- 172.21.15.103:0/3625235269 learned_addr learned my addr 172.21.15.103:0/3625235269 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71be00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc71c104490 0x7fc71c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71b400640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102980 0x7fc71c19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 -- 172.21.15.103:0/3625235269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc71c104490 msgr2=0x7fc71c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc71c104490 0x7fc71c19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 -- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102980 msgr2=0x7fc71c19b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102980 0x7fc71c19b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.150 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 -- 172.21.15.103:0/3625235269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c19cb40 con 0x7fc71c103ad0 2024-09-11T00:29:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71be00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc71c104490 0x7fc71c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.146+0000 7fc71aa00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c103ad0 0x7fc71c19b650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc708009a50 tx=0x7fc7080042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.147+0000 7fc713e00640 1 -- 172.21.15.103:0/3625235269 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70803f030 con 0x7fc71c103ad0 2024-09-11T00:29:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.147+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c19cdd0 con 0x7fc71c103ad0 2024-09-11T00:29:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.147+0000 7fc713e00640 1 -- 172.21.15.103:0/3625235269 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708031910 con 0x7fc71c103ad0 2024-09-11T00:29:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.147+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c1a3500 con 0x7fc71c103ad0 2024-09-11T00:29:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.147+0000 7fc713e00640 1 -- 172.21.15.103:0/3625235269 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708049060 con 0x7fc71c103ad0 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.150+0000 7fc7253e6740 1 Processor -- start 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc713e00640 1 -- 172.21.15.103:0/3625235269 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc70803ba40 con 0x7fc71c103ad0 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc7253e6740 1 -- start start 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e14266e790 0x55e142670b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1426710c0 0x55e1426734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142675fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc7253e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e1425e16b0 con 0x55e142673b00 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc7253e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e1425d0ae0 con 0x55e1426710c0 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc7253e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1425d48f0 con 0x55e14266e790 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc71be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e14266e790 0x55e142670b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc71b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1426710c0 0x55e1426734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc71be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e14266e790 0x55e142670b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57644/0 (socket says 172.21.15.103:57644) 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc71be00640 1 -- 172.21.15.103:0/3403163764 learned_addr learned my addr 172.21.15.103:0/3403163764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.151+0000 7fc71aa00640 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142675fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc71b400640 1 -- 172.21.15.103:0/3403163764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e14266e790 msgr2=0x55e142670b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc71b400640 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e14266e790 0x55e142670b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc71b400640 1 -- 172.21.15.103:0/3403163764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 msgr2=0x55e142675fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc71b400640 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142675fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc71b400640 1 -- 172.21.15.103:0/3403163764 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14259d780 con 0x55e1426710c0 2024-09-11T00:29:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc71b400640 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1426710c0 0x55e1426734b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009c20 tx=0x7fc70c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90be94bb5d35d41 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc710c00640 1 -- 172.21.15.103:0/3403163764 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c01e070 con 0x55e1426710c0 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc713e00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7000b52e0 0x7fc7000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc710c00640 1 -- 172.21.15.103:0/3403163764 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c009dd0 con 0x55e1426710c0 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc710c00640 1 -- 172.21.15.103:0/3403163764 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0187d0 con 0x55e1426710c0 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc713e00640 1 -- 172.21.15.103:0/3625235269 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc708106ba0 con 0x7fc71c103ad0 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.152+0000 7fc71be00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7000b52e0 0x7fc7000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 -- 172.21.15.103:0/3403163764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1426710c0 msgr2=0x55e1426734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1426710c0 0x55e1426734b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009c20 tx=0x7fc70c011810 comp rx=0 tx=0).stop 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc71be00640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7000b52e0 0x7fc7000b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc714009fd0 tx=0x7fc714009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 -- 172.21.15.103:0/3403163764 shutdown_connections 2024-09-11T00:29:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142675fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e1426710c0 0x55e1426734b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 --2- 172.21.15.103:0/3403163764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e14266e790 0x55e142670b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 -- 172.21.15.103:0/3403163764 >> 172.21.15.103:0/3403163764 conn(0x55e1426698e0 msgr2=0x55e14266a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 -- 172.21.15.103:0/3403163764 shutdown_connections 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.153+0000 7fc7253e6740 1 -- 172.21.15.103:0/3403163764 wait complete. 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.154+0000 7fc7253e6740 1 -- start start 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.154+0000 7fc7253e6740 1 Processor -- start 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142677e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142678530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142679100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc7253e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e142508ab0 con 0x55e142673b00 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc7253e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55e142577570 con 0x55e14266e790 2024-09-11T00:29:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc7253e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e142676ae0 con 0x55e1426710c0 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc7253e6740 10 client.0 _ll_register_callbacks cb 0x55e142586100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142677e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142678530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142677e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:49930/0 (socket says 172.21.15.103:49930) 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 -- 172.21.15.103:0/146079311 learned_addr learned my addr 172.21.15.103:0/146079311 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71be00640 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142679100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 -- 172.21.15.103:0/146079311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 msgr2=0x55e142678530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142678530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 -- 172.21.15.103:0/146079311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 msgr2=0x55e142679100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142679100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71b400640 1 -- 172.21.15.103:0/146079311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e142332790 con 0x55e14266e790 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.155+0000 7fc71aa00640 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142678530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc71b400640 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142677e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009bf0 tx=0x7fc70c009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=692532516d81f428 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc705600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc705600640 1 -- 172.21.15.103:0/146079311 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70c01e040 con 0x55e14266e790 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc705600640 1 -- 172.21.15.103:0/146079311 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc70c0050c0 con 0x55e14266e790 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70c009dd0 con 0x55e14266e790 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc705600640 1 -- 172.21.15.103:0/146079311 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc70c032050 con 0x55e14266e790 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc7253e6740 10 client.26102 fetch_fsmap learned FSMap version 43 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e1423ab620 con 0x55e14266e790 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc705600640 1 -- 172.21.15.103:0/146079311 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70c009dd0 con 0x55e14266e790 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.156+0000 7fc705600640 1 -- 172.21.15.103:0/146079311 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc70c021030 con 0x55e14266e790 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc7253e6740 10 client.26102 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc7253e6740 10 client.26102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc7253e6740 20 client.26102 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc7253e6740 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e142683550 0x55e142685960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc7253e6740 4 client.26102 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:29:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1425e3600 con 0x55e142683550 2024-09-11T00:29:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc71aa00640 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e142683550 0x55e142685960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc705600640 10 client.26102 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.157+0000 7fc71aa00640 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e142683550 0x55e142685960 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=393212c128132e67 server_cookie=189190d49730f5c7 in_seq=0 out_seq=0 2024-09-11T00:29:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.158+0000 7fc705600640 1 -- 172.21.15.103:0/146079311 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e1425e3600 con 0x55e142683550 2024-09-11T00:29:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.158+0000 7fc705600640 10 client.26102 handle_command_reply: tid=0 2024-09-11T00:29:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.158+0000 7fc7253e6740 1 client.26102 shutdown 2024-09-11T00:29:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 msgr2=0x55e142677e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142677e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009bf0 tx=0x7fc70c009870 comp rx=0 tx=0).stop 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 shutdown_connections 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e142679100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e142683550 0x55e142685960 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142678530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc705600640 0 client.26102 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 --2- 172.21.15.103:0/146079311 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142677e90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 >> 172.21.15.103:0/146079311 conn(0x55e1426698e0 msgr2=0x55e142668a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 shutdown_connections 2024-09-11T00:29:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 1 -- 172.21.15.103:0/146079311 wait complete. 2024-09-11T00:29:54.166 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.159+0000 7fc7253e6740 20 client.26102 trim_cache size 0 max 16384 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 Processor -- start 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 -- start start 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142676ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142677420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e14266b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c1b01a0 con 0x55e142673b00 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c1ada70 con 0x55e14266e790 2024-09-11T00:29:54.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc7253e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c1ab8c0 con 0x55e1426710c0 2024-09-11T00:29:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc71aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142677420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc71b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142676ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc71aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142677420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57654/0 (socket says 172.21.15.103:57654) 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc71b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142676ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:49944/0 (socket says 172.21.15.103:49944) 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc71aa00640 1 -- 172.21.15.103:0/640229828 learned_addr learned my addr 172.21.15.103:0/640229828 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.180+0000 7fc71be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e14266b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc71aa00640 1 -- 172.21.15.103:0/640229828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 msgr2=0x55e142676ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc71aa00640 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142676ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc71aa00640 1 -- 172.21.15.103:0/640229828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 msgr2=0x55e14266b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc71aa00640 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e14266b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc71aa00640 1 -- 172.21.15.103:0/640229828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e142670ef0 con 0x55e1426710c0 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc71b400640 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142676ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc71aa00640 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142677420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc7080f0170 tx=0x7fc7080b4040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92f125b3e002644d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc707e00640 1 -- 172.21.15.103:0/640229828 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708035040 con 0x55e1426710c0 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc707e00640 1 -- 172.21.15.103:0/640229828 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70810c070 con 0x55e1426710c0 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.181+0000 7fc707e00640 1 -- 172.21.15.103:0/640229828 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7080cd020 con 0x55e1426710c0 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 -- 172.21.15.103:0/640229828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 msgr2=0x55e142677420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142677420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc7080f0170 tx=0x7fc7080b4040 comp rx=0 tx=0).stop 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 -- 172.21.15.103:0/640229828 shutdown_connections 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e142673b00 0x55e14266b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1426710c0 0x55e142677420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 --2- 172.21.15.103:0/640229828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e142676ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 -- 172.21.15.103:0/640229828 >> 172.21.15.103:0/640229828 conn(0x55e1426698e0 msgr2=0x55e14266dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 -- 172.21.15.103:0/640229828 shutdown_connections 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 -- 172.21.15.103:0/640229828 wait complete. 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.182+0000 7fc7253e6740 1 -- start start 2024-09-11T00:29:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 1 Processor -- start 2024-09-11T00:29:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e14267eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1426710c0 0x55e14267f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e142673b00 0x55e14267d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c1abf30 con 0x55e1426710c0 2024-09-11T00:29:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c1ad3a0 con 0x55e14266e790 2024-09-11T00:29:54.189 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c1ac600 con 0x55e142673b00 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc7253e6740 10 client.0 _ll_register_callbacks cb 0x7fc70c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc71be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e142673b00 0x55e14267d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc71be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e142673b00 0x55e14267d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:57656/0 (socket says 172.21.15.103:57656) 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc71be00640 1 -- 172.21.15.103:0/4050876146 learned_addr learned my addr 172.21.15.103:0/4050876146 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc71aa00640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1426710c0 0x55e14267f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.183+0000 7fc71b400640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e14267eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc71be00640 1 -- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 msgr2=0x55e14267eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc71be00640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e14267eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc71be00640 1 -- 172.21.15.103:0/4050876146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1426710c0 msgr2=0x55e14267f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc71be00640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1426710c0 0x55e14267f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc71be00640 1 -- 172.21.15.103:0/4050876146 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14267de00 con 0x55e142673b00 2024-09-11T00:29:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc71b400640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e14267eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc71be00640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e142673b00 0x55e14267d8c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc714011d10 tx=0x7fc714004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c8dd152531e257a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc705600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc705600640 1 -- 172.21.15.103:0/4050876146 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc714011020 con 0x55e142673b00 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc70810c070 con 0x55e142673b00 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.184+0000 7fc705600640 1 -- 172.21.15.103:0/4050876146 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc7140111c0 con 0x55e142673b00 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc705600640 1 -- 172.21.15.103:0/4050876146 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc714037050 con 0x55e142673b00 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 10 client.26128 fetch_fsmap learned FSMap version 43 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e1426f0cc0 con 0x55e142673b00 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc705600640 1 -- 172.21.15.103:0/4050876146 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc70810c070 con 0x55e142673b00 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc705600640 1 -- 172.21.15.103:0/4050876146 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc714032b80 con 0x55e142673b00 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 10 client.26128 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:29:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:29:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 20 client.26128 populate_metadata read hostname 'smithi103' 2024-09-11T00:29:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e1426f1720 0x55e1426f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:29:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 4 client.26128 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:29:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1425df0b0 con 0x55e1426f1720 2024-09-11T00:29:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.185+0000 7fc71b400640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e1426f1720 0x55e1426f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:29:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.186+0000 7fc71b400640 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e1426f1720 0x55e1426f3b40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e89e14082443b5 server_cookie=1b864e762ef9afa4 in_seq=0 out_seq=0 2024-09-11T00:29:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.186+0000 7fc705600640 10 client.26128 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.196+0000 7fc705600640 1 -- 172.21.15.103:0/4050876146 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1425df0b0 con 0x55e1426f1720 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.196+0000 7fc705600640 10 client.26128 handle_command_reply: tid=0 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.196+0000 7fc7253e6740 1 client.26128 shutdown 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e142673b00 msgr2=0x55e14267d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e142673b00 0x55e14267d8c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc714011d10 tx=0x7fc714004a40 comp rx=0 tx=0).stop 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 shutdown_connections 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55e1426f1720 0x55e1426f3b40 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e142673b00 0x55e14267d8c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1426710c0 0x55e14267f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc705600640 0 client.26128 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 --2- 172.21.15.103:0/4050876146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e14266e790 0x55e14267eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 >> 172.21.15.103:0/4050876146 conn(0x55e1426698e0 msgr2=0x55e142676ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 shutdown_connections 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 1 -- 172.21.15.103:0/4050876146 wait complete. 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.197+0000 7fc7253e6740 20 client.26128 trim_cache size 0 max 16384 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:29:54.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:29:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:29:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:29:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7000b52e0 msgr2=0x7fc7000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7000b52e0 0x7fc7000b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc714009fd0 tx=0x7fc714009500 comp rx=0 tx=0).stop 2024-09-11T00:29:54.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c103ad0 msgr2=0x7fc71c19b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c103ad0 0x7fc71c19b650 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc708009a50 tx=0x7fc7080042f0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 shutdown_connections 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7000b52e0 0x7fc7000b77a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc71c104490 0x7fc71c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc71c103ad0 0x7fc71c19b650 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 --2- 172.21.15.103:0/3625235269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc71c102980 0x7fc71c19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 >> 172.21.15.103:0/3625235269 conn(0x7fc71c0fe240 msgr2=0x7fc71c109260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 shutdown_connections 2024-09-11T00:29:54.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:29:54.198+0000 7fc722400640 1 -- 172.21.15.103:0/3625235269 wait complete. 2024-09-11T00:29:54.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:54 smithi183 ceph-mon[35934]: pgmap v604: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 389 op/s 2024-09-11T00:29:54.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:54 smithi103 ceph-mon[30578]: pgmap v604: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 389 op/s 2024-09-11T00:29:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:54 smithi117 ceph-mon[35525]: pgmap v604: 129 pgs: 129 active+clean; 77 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 389 op/s 2024-09-11T00:29:56.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:56 smithi183 ceph-mon[35934]: pgmap v605: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 495 op/s 2024-09-11T00:29:56.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:56 smithi103 ceph-mon[30578]: pgmap v605: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 495 op/s 2024-09-11T00:29:56.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:56 smithi117 ceph-mon[35525]: pgmap v605: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.2 MiB/s wr, 495 op/s 2024-09-11T00:29:58.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:29:58 smithi183 ceph-mon[35934]: pgmap v606: 129 pgs: 129 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1010 KiB/s wr, 413 op/s 2024-09-11T00:29:58.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:29:58 smithi103 ceph-mon[30578]: pgmap v606: 129 pgs: 129 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1010 KiB/s wr, 413 op/s 2024-09-11T00:29:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:29:58 smithi117 ceph-mon[35525]: pgmap v606: 129 pgs: 129 active+clean; 82 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1010 KiB/s wr, 413 op/s 2024-09-11T00:30:00.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:00 smithi183 ceph-mon[35934]: pgmap v607: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 474 op/s 2024-09-11T00:30:00.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:00 smithi183 ceph-mon[35934]: overall HEALTH_OK 2024-09-11T00:30:00.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:00 smithi103 ceph-mon[30578]: pgmap v607: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 474 op/s 2024-09-11T00:30:00.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:00 smithi103 ceph-mon[30578]: overall HEALTH_OK 2024-09-11T00:30:00.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:00 smithi117 ceph-mon[35525]: pgmap v607: 129 pgs: 129 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.1 MiB/s wr, 474 op/s 2024-09-11T00:30:00.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:00 smithi117 ceph-mon[35525]: overall HEALTH_OK 2024-09-11T00:30:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:02 smithi183 ceph-mon[35934]: pgmap v608: 129 pgs: 129 active+clean; 76 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 477 op/s 2024-09-11T00:30:02.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:02 smithi103 ceph-mon[30578]: pgmap v608: 129 pgs: 129 active+clean; 76 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 477 op/s 2024-09-11T00:30:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:02 smithi117 ceph-mon[35525]: pgmap v608: 129 pgs: 129 active+clean; 76 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 477 op/s 2024-09-11T00:30:03.921 INFO:tasks.workunit.client.0.smithi103.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-09-11T00:30:04.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:04 smithi183 ceph-mon[35934]: pgmap v609: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-09-11T00:30:04.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:04 smithi103 ceph-mon[30578]: pgmap v609: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-09-11T00:30:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:04 smithi117 ceph-mon[35525]: pgmap v609: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.0 MiB/s wr, 435 op/s 2024-09-11T00:30:06.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:06 smithi183 ceph-mon[35934]: pgmap v610: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 535 op/s 2024-09-11T00:30:06.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:06 smithi103 ceph-mon[30578]: pgmap v610: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 535 op/s 2024-09-11T00:30:06.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:06 smithi117 ceph-mon[35525]: pgmap v610: 129 pgs: 129 active+clean; 81 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 535 op/s 2024-09-11T00:30:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.330+0000 7f2558a00640 1 -- 172.21.15.103:0/4027965231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2554102790 msgr2=0x7f2554102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.330+0000 7f2558a00640 1 --2- 172.21.15.103:0/4027965231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2554102790 0x7f2554102b90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2544009930 tx=0x7f254402a420 comp rx=0 tx=0).stop 2024-09-11T00:30:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.331+0000 7f2558a00640 1 -- 172.21.15.103:0/4027965231 shutdown_connections 2024-09-11T00:30:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.331+0000 7f2558a00640 1 --2- 172.21.15.103:0/4027965231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f25541042a0 0x7f255410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.331+0000 7f2558a00640 1 --2- 172.21.15.103:0/4027965231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25541038e0 0x7f2554103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.331+0000 7f2558a00640 1 --2- 172.21.15.103:0/4027965231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2554102790 0x7f2554102b90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.331+0000 7f2558a00640 1 -- 172.21.15.103:0/4027965231 >> 172.21.15.103:0/4027965231 conn(0x7f25540fe030 msgr2=0x7f2554100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.331+0000 7f2558a00640 1 -- 172.21.15.103:0/4027965231 shutdown_connections 2024-09-11T00:30:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.331+0000 7f2558a00640 1 -- 172.21.15.103:0/4027965231 wait complete. 2024-09-11T00:30:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 Processor -- start 2024-09-11T00:30:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 -- start start 2024-09-11T00:30:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2554102790 0x7f2554076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25541038e0 0x7f2554075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25541042a0 0x7f2554075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f255410e450 con 0x7f25541038e0 2024-09-11T00:30:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f255410e2d0 con 0x7f2554102790 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2558a00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f255410e5d0 con 0x7f25541042a0 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2552800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2554102790 0x7f2554076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f254be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25541038e0 0x7f2554075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2552800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2554102790 0x7f2554076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42862/0 (socket says 172.21.15.103:42862) 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2552800640 1 -- 172.21.15.103:0/1918412594 learned_addr learned my addr 172.21.15.103:0/1918412594 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.332+0000 7f2553200640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25541042a0 0x7f2554075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2552800640 1 -- 172.21.15.103:0/1918412594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25541042a0 msgr2=0x7f2554075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2552800640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25541042a0 0x7f2554075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2552800640 1 -- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25541038e0 msgr2=0x7f2554075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2552800640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25541038e0 0x7f2554075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2552800640 1 -- 172.21.15.103:0/1918412594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25540762f0 con 0x7f2554102790 2024-09-11T00:30:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f254be00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25541038e0 0x7f2554075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2552800640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2554102790 0x7f2554076ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2544031500 tx=0x7f2544003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f254b400640 1 -- 172.21.15.103:0/1918412594 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254403f070 con 0x7f2554102790 2024-09-11T00:30:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f254b400640 1 -- 172.21.15.103:0/1918412594 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2544031c50 con 0x7f2554102790 2024-09-11T00:30:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f254b400640 1 -- 172.21.15.103:0/1918412594 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544035030 con 0x7f2554102790 2024-09-11T00:30:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25541a2c60 con 0x7f2554102790 2024-09-11T00:30:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.333+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25541a30d0 con 0x7f2554102790 2024-09-11T00:30:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 Processor -- start 2024-09-11T00:30:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 -- start start 2024-09-11T00:30:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x55786569cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569d2a0 0x55786569f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55786560d6b0 con 0x55786569d2a0 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5578655fcae0 con 0x55786569a970 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.337+0000 7f255c282740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578656008f0 con 0x55786569fce0 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:58622/0 (socket says 172.21.15.103:58622) 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 -- 172.21.15.103:0/3707146747 learned_addr learned my addr 172.21.15.103:0/3707146747 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254b400640 1 -- 172.21.15.103:0/1918412594 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f254403b460 con 0x7f2554102790 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f2553200640 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x55786569cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 -- 172.21.15.103:0/3707146747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 msgr2=0x55786569cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x55786569cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 -- 172.21.15.103:0/3707146747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569d2a0 msgr2=0x55786569f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569d2a0 0x55786569f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 -- 172.21.15.103:0/3707146747 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578655c9780 con 0x55786569fce0 2024-09-11T00:30:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.338+0000 7f254be00640 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a21a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f253c0115f0 tx=0x7f253c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5696eecd43d58f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.339+0000 7f2543400640 1 -- 172.21.15.103:0/3707146747 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c01e070 con 0x55786569fce0 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.339+0000 7f2543400640 1 -- 172.21.15.103:0/3707146747 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c00faf0 con 0x55786569fce0 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.339+0000 7f2543400640 1 -- 172.21.15.103:0/3707146747 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c019b60 con 0x55786569fce0 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.339+0000 7f255c282740 1 -- 172.21.15.103:0/3707146747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 msgr2=0x5578656a21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.339+0000 7f255c282740 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a21a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f253c0115f0 tx=0x7f253c011ab0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f255c282740 1 -- 172.21.15.103:0/3707146747 shutdown_connections 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f255c282740 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a21a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f255c282740 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569d2a0 0x55786569f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f255c282740 1 --2- 172.21.15.103:0/3707146747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x55786569cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f255c282740 1 -- 172.21.15.103:0/3707146747 >> 172.21.15.103:0/3707146747 conn(0x557865695ac0 msgr2=0x557865696850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f255c282740 1 -- 172.21.15.103:0/3707146747 shutdown_connections 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f254b400640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25380b51f0 0x7f25380b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f255c282740 1 -- 172.21.15.103:0/3707146747 wait complete. 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f254b400640 1 -- 172.21.15.103:0/1918412594 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f2544106920 con 0x7f2554102790 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.340+0000 7f254be00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25380b51f0 0x7f25380b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 -- start start 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 Processor -- start 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f254be00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25380b51f0 0x7f25380b76b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f253c0115c0 tx=0x7f253c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656a4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569d2a0 0x5578656a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557865534ab0 con 0x55786569a970 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5578655a3570 con 0x55786569d2a0 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.341+0000 7f255c282740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5578656a2cc0 con 0x55786569fce0 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f255c282740 10 client.0 _ll_register_callbacks cb 0x5578655b2100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2553200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36862/0 (socket says 172.21.15.103:36862) 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 -- 172.21.15.103:0/4036293849 learned_addr learned my addr 172.21.15.103:0/4036293849 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:07.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f254be00640 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569d2a0 0x5578656a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 -- 172.21.15.103:0/4036293849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 msgr2=0x5578656a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 -- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569d2a0 msgr2=0x5578656a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569d2a0 0x5578656a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2552800640 1 -- 172.21.15.103:0/4036293849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55786535e790 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.342+0000 7f2553200640 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.343+0000 7f2552800640 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656a4160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f25440ca070 tx=0x7f25440c8db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314eba0109f0fd2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.343+0000 7f2537e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.343+0000 7f2537e00640 1 -- 172.21.15.103:0/4036293849 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2544049040 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.343+0000 7f2537e00640 1 -- 172.21.15.103:0/4036293849 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f25440d5e50 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.344+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253c00faf0 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.344+0000 7f2537e00640 1 -- 172.21.15.103:0/4036293849 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f25440b42d0 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.344+0000 7f255c282740 10 client.16065 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.344+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5578653d7620 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.344+0000 7f2537e00640 1 -- 172.21.15.103:0/4036293849 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c00faf0 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.344+0000 7f2537e00640 1 -- 172.21.15.103:0/4036293849 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f25440cc060 con 0x55786569a970 2024-09-11T00:30:07.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.345+0000 7f255c282740 10 client.16065 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.345+0000 7f255c282740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.345+0000 7f255c282740 20 client.16065 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.345+0000 7f255c282740 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5578656af920 0x5578656b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.345+0000 7f255c282740 4 client.16065 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.345+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55786560f600 con 0x5578656af920 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.345+0000 7f254be00640 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5578656af920 0x5578656b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.346+0000 7f254be00640 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5578656af920 0x5578656b1d30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b19ec68797a29ef server_cookie=9d6db0e2a33bb7a5 in_seq=0 out_seq=0 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.346+0000 7f2537e00640 10 client.16065 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.346+0000 7f2537e00640 1 -- 172.21.15.103:0/4036293849 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55786560f600 con 0x5578656af920 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.346+0000 7f2537e00640 10 client.16065 handle_command_reply: tid=0 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.346+0000 7f255c282740 1 client.16065 shutdown 2024-09-11T00:30:07.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 msgr2=0x5578656a4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656a4160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f25440ca070 tx=0x7f25440c8db0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 shutdown_connections 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5578656af920 0x5578656b1d30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569fce0 0x5578656a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f2537e00640 0 client.16065 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569d2a0 0x5578656a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 --2- 172.21.15.103:0/4036293849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656a4160 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 >> 172.21.15.103:0/4036293849 conn(0x557865695ac0 msgr2=0x557865694bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 shutdown_connections 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 1 -- 172.21.15.103:0/4036293849 wait complete. 2024-09-11T00:30:07.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.347+0000 7f255c282740 20 client.16065 trim_cache size 0 max 16384 2024-09-11T00:30:07.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.369+0000 7f255c282740 1 Processor -- start 2024-09-11T00:30:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.370+0000 7f255c282740 1 -- start start 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.371+0000 7f255c282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x5578656a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.371+0000 7f255c282740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x557865696510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.371+0000 7f255c282740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569fce0 0x557865696a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.371+0000 7f255c282740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25541affd0 con 0x55786569fce0 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.371+0000 7f255c282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25541ad8a0 con 0x55786569a970 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.371+0000 7f255c282740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25541ab6f0 con 0x55786569d2a0 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.371+0000 7f2552800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x5578656a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f2552800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x5578656a3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42872/0 (socket says 172.21.15.103:42872) 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f2552800640 1 -- 172.21.15.103:0/1961502895 learned_addr learned my addr 172.21.15.103:0/1961502895 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f254be00640 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x557865696510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f254be00640 1 -- 172.21.15.103:0/1961502895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 msgr2=0x5578656a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f254be00640 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x5578656a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f254be00640 1 -- 172.21.15.103:0/1961502895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569fce0 msgr2=0x557865696a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f254be00640 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569fce0 0x557865696a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.372+0000 7f254be00640 1 -- 172.21.15.103:0/1961502895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578656a5740 con 0x55786569d2a0 2024-09-11T00:30:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.373+0000 7f254be00640 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x557865696510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5578656b2580 tx=0x7f253c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b891cb0af50e1cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.374+0000 7f2542a00640 1 -- 172.21.15.103:0/1961502895 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c01e040 con 0x55786569d2a0 2024-09-11T00:30:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.374+0000 7f2542a00640 1 -- 172.21.15.103:0/1961502895 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c03d070 con 0x55786569d2a0 2024-09-11T00:30:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.374+0000 7f2542a00640 1 -- 172.21.15.103:0/1961502895 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c00f9c0 con 0x55786569d2a0 2024-09-11T00:30:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.375+0000 7f255c282740 1 -- 172.21.15.103:0/1961502895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 msgr2=0x557865696510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.375+0000 7f255c282740 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x557865696510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x5578656b2580 tx=0x7f253c020000 comp rx=0 tx=0).stop 2024-09-11T00:30:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.376+0000 7f255c282740 1 -- 172.21.15.103:0/1961502895 shutdown_connections 2024-09-11T00:30:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.376+0000 7f255c282740 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569fce0 0x557865696a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.376+0000 7f255c282740 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x557865696510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.376+0000 7f255c282740 1 --2- 172.21.15.103:0/1961502895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569a970 0x5578656a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.376+0000 7f255c282740 1 -- 172.21.15.103:0/1961502895 >> 172.21.15.103:0/1961502895 conn(0x557865695ac0 msgr2=0x557865699ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.376+0000 7f255c282740 1 -- 172.21.15.103:0/1961502895 shutdown_connections 2024-09-11T00:30:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.376+0000 7f255c282740 1 -- 172.21.15.103:0/1961502895 wait complete. 2024-09-11T00:30:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.377+0000 7f255c282740 1 -- start start 2024-09-11T00:30:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.377+0000 7f255c282740 1 Processor -- start 2024-09-11T00:30:07.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f255c282740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f255c282740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x5578656aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f255c282740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569fce0 0x5578656ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f255c282740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25541abd60 con 0x55786569a970 2024-09-11T00:30:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f255c282740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f25541ad1d0 con 0x55786569fce0 2024-09-11T00:30:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f255c282740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25541ac430 con 0x55786569d2a0 2024-09-11T00:30:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f255c282740 10 client.0 _ll_register_callbacks cb 0x7f2544005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f254be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x5578656aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f254be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x5578656aaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:58642/0 (socket says 172.21.15.103:58642) 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f254be00640 1 -- 172.21.15.103:0/1446820240 learned_addr learned my addr 172.21.15.103:0/1446820240 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f2553200640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569fce0 0x5578656ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f2552800640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f254be00640 1 -- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569fce0 msgr2=0x5578656ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f254be00640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569fce0 0x5578656ab0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.378+0000 7f254be00640 1 -- 172.21.15.103:0/1446820240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 msgr2=0x5578656abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f254be00640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.382 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f254be00640 1 -- 172.21.15.103:0/1446820240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578656a9560 con 0x55786569d2a0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f2552800640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f254be00640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x5578656aaa40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f253c019680 tx=0x7f253c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e795306ac8ea3df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f2537e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f2537e00640 1 -- 172.21.15.103:0/1446820240 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253c01e050 con 0x55786569d2a0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f253c03d070 con 0x55786569d2a0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.379+0000 7f2537e00640 1 -- 172.21.15.103:0/1446820240 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f253c01ada0 con 0x55786569d2a0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f2537e00640 1 -- 172.21.15.103:0/1446820240 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f253c04f050 con 0x55786569d2a0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 10 client.26146 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5578657251d0 con 0x55786569d2a0 2024-09-11T00:30:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f2537e00640 1 -- 172.21.15.103:0/1446820240 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f253c03d070 con 0x55786569d2a0 2024-09-11T00:30:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f2537e00640 1 -- 172.21.15.103:0/1446820240 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f253c021030 con 0x55786569d2a0 2024-09-11T00:30:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 10 client.26146 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 10 client.26146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 20 client.26146 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557865725950 0x557865727d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 4 client.26146 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55786560b0b0 con 0x557865725950 2024-09-11T00:30:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.380+0000 7f2552800640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557865725950 0x557865727d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.381+0000 7f2552800640 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557865725950 0x557865727d70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebee23530f03c29 server_cookie=c08f7d7316af7240 in_seq=0 out_seq=0 2024-09-11T00:30:07.385 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.381+0000 7f2537e00640 10 client.26146 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f2537e00640 1 -- 172.21.15.103:0/1446820240 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55786560b0b0 con 0x557865725950 2024-09-11T00:30:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f2537e00640 10 client.26146 handle_command_reply: tid=0 2024-09-11T00:30:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 client.26146 shutdown 2024-09-11T00:30:07.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 msgr2=0x5578656aaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x5578656aaa40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f253c019680 tx=0x7f253c005f90 comp rx=0 tx=0).stop 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 shutdown_connections 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55786569fce0 0x5578656ab0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x557865725950 0x557865727d70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55786569d2a0 0x5578656aaa40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 --2- 172.21.15.103:0/1446820240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55786569a970 0x5578656abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f2537e00640 0 client.26146 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 >> 172.21.15.103:0/1446820240 conn(0x557865695ac0 msgr2=0x5578656a00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 shutdown_connections 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 1 -- 172.21.15.103:0/1446820240 wait complete. 2024-09-11T00:30:07.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.394+0000 7f255c282740 20 client.26146 trim_cache size 0 max 16384 2024-09-11T00:30:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:30:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:30:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:30:07.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25380b51f0 msgr2=0x7f25380b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25380b51f0 0x7f25380b76b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f253c0115c0 tx=0x7f253c000bc0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2554102790 msgr2=0x7f2554076ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2554102790 0x7f2554076ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2544031500 tx=0x7f2544003c10 comp rx=0 tx=0).stop 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 shutdown_connections 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f25380b51f0 0x7f25380b76b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25541042a0 0x7f2554075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25541038e0 0x7f2554075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 --2- 172.21.15.103:0/1918412594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2554102790 0x7f2554076ce0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 >> 172.21.15.103:0/1918412594 conn(0x7f25540fe030 msgr2=0x7f2554106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 shutdown_connections 2024-09-11T00:30:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:07.395+0000 7f2558a00640 1 -- 172.21.15.103:0/1918412594 wait complete. 2024-09-11T00:30:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:08 smithi183 ceph-mon[35934]: pgmap v611: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 418 op/s 2024-09-11T00:30:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:08 smithi103 ceph-mon[30578]: pgmap v611: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 418 op/s 2024-09-11T00:30:08.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:08 smithi117 ceph-mon[35525]: pgmap v611: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.1 MiB/s wr, 418 op/s 2024-09-11T00:30:10.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:10 smithi183 ceph-mon[35934]: pgmap v612: 129 pgs: 129 active+clean; 80 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-11T00:30:10.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:10 smithi103 ceph-mon[30578]: pgmap v612: 129 pgs: 129 active+clean; 80 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-11T00:30:10.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:10 smithi117 ceph-mon[35525]: pgmap v612: 129 pgs: 129 active+clean; 80 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.2 MiB/s wr, 466 op/s 2024-09-11T00:30:12.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:12 smithi183 ceph-mon[35934]: pgmap v613: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 484 op/s 2024-09-11T00:30:12.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:12 smithi103 ceph-mon[30578]: pgmap v613: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 484 op/s 2024-09-11T00:30:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:12 smithi117 ceph-mon[35525]: pgmap v613: 129 pgs: 129 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 484 op/s 2024-09-11T00:30:14.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:14 smithi103 ceph-mon[30578]: pgmap v614: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-11T00:30:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:14 smithi117 ceph-mon[35525]: pgmap v614: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-11T00:30:14.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:14 smithi183 ceph-mon[35934]: pgmap v614: 129 pgs: 129 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 1.1 MiB/s wr, 420 op/s 2024-09-11T00:30:16.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:16 smithi103 ceph-mon[30578]: pgmap v615: 129 pgs: 129 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-09-11T00:30:16.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:16 smithi117 ceph-mon[35525]: pgmap v615: 129 pgs: 129 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-09-11T00:30:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:16 smithi183 ceph-mon[35934]: pgmap v615: 129 pgs: 129 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 555 op/s 2024-09-11T00:30:18.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:18 smithi103 ceph-mon[30578]: pgmap v616: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 417 op/s 2024-09-11T00:30:18.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:18 smithi117 ceph-mon[35525]: pgmap v616: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 417 op/s 2024-09-11T00:30:18.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:18 smithi183 ceph-mon[35934]: pgmap v616: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 417 op/s 2024-09-11T00:30:19.017 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x63e5d2a 2024-09-11T00:30:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 -- 172.21.15.103:0/363004519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 msgr2=0x7fd774103d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 --2- 172.21.15.103:0/363004519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 0x7fd774103d60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd760009930 tx=0x7fd76002a440 comp rx=0 tx=0).stop 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 -- 172.21.15.103:0/363004519 shutdown_connections 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 --2- 172.21.15.103:0/363004519 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 0x7fd77410ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 --2- 172.21.15.103:0/363004519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 0x7fd774103d60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 --2- 172.21.15.103:0/363004519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd774102790 0x7fd774102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 -- 172.21.15.103:0/363004519 >> 172.21.15.103:0/363004519 conn(0x7fd7740fe030 msgr2=0x7fd774100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 -- 172.21.15.103:0/363004519 shutdown_connections 2024-09-11T00:30:20.562 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.560+0000 7fd77ae00640 1 -- 172.21.15.103:0/363004519 wait complete. 2024-09-11T00:30:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.561+0000 7fd77ae00640 1 Processor -- start 2024-09-11T00:30:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77ae00640 1 -- start start 2024-09-11T00:30:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd774102790 0x7fd774198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77ae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 0x7fd774199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77ae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 0x7fd774199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410e400 con 0x7fd774102790 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77ae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410e280 con 0x7fd7741042a0 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77ae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd77410e580 con 0x7fd7741038e0 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd779000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 0x7fd774199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 0x7fd774199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd779a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd774102790 0x7fd774198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77a400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 0x7fd774199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:45434/0 (socket says 172.21.15.103:45434) 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd779000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 0x7fd774199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44640/0 (socket says 172.21.15.103:44640) 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.562+0000 7fd77a400640 1 -- 172.21.15.103:0/3681600605 learned_addr learned my addr 172.21.15.103:0/3681600605 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77a400640 1 -- 172.21.15.103:0/3681600605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 msgr2=0x7fd774199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77a400640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 0x7fd774199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77a400640 1 -- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd774102790 msgr2=0x7fd774198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77a400640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd774102790 0x7fd774198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77a400640 1 -- 172.21.15.103:0/3681600605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77419a6f0 con 0x7fd7741042a0 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77a400640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 0x7fd774199e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00eb10 tx=0x7fd76c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd772000640 1 -- 172.21.15.103:0/3681600605 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c004b20 con 0x7fd7741042a0 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd772000640 1 -- 172.21.15.103:0/3681600605 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd76c004cc0 con 0x7fd7741042a0 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77419a9e0 con 0x7fd7741042a0 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd772000640 1 -- 172.21.15.103:0/3681600605 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c010670 con 0x7fd7741042a0 2024-09-11T00:30:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.563+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7741a10b0 con 0x7fd7741042a0 2024-09-11T00:30:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.567+0000 7fd772000640 1 -- 172.21.15.103:0/3681600605 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd76c005020 con 0x7fd7741042a0 2024-09-11T00:30:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.567+0000 7fd77f21b740 1 Processor -- start 2024-09-11T00:30:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.567+0000 7fd77f21b740 1 -- start start 2024-09-11T00:30:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.568+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b707275d80 0x55b707278170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.568+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.568+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 0x55b70727d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.568+0000 7fd77f21b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b7071e46b0 con 0x55b707275d80 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.568+0000 7fd77f21b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b7071d3ae0 con 0x55b70727b0f0 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.568+0000 7fd77f21b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b7071d78f0 con 0x55b7072786b0 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.569+0000 7fd772000640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd75c0b52e0 0x7fd75c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.569+0000 7fd772000640 1 -- 172.21.15.103:0/3681600605 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fd76c0e64c0 con 0x7fd7741042a0 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.569+0000 7fd77a400640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd75c0b52e0 0x7fd75c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.569+0000 7fd779000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b707275d80 0x55b707278170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.569+0000 7fd779000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b707275d80 0x55b707278170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57348/0 (socket says 172.21.15.103:57348) 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.569+0000 7fd779000640 1 -- 172.21.15.103:0/2379310498 learned_addr learned my addr 172.21.15.103:0/2379310498 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.569+0000 7fd779a00640 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd779000640 1 -- 172.21.15.103:0/2379310498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 msgr2=0x55b70727aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd779000640 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd779000640 1 -- 172.21.15.103:0/2379310498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 msgr2=0x55b70727d5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd779000640 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 0x55b70727d5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd779000640 1 -- 172.21.15.103:0/2379310498 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7071a0780 con 0x55b707275d80 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd77a400640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd75c0b52e0 0x7fd75c0b77a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0a8ec0 tx=0x7fd76c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd779000640 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b707275d80 0x55b707278170 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd76002a960 tx=0x7fd760005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc520341078ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd766a00640 1 -- 172.21.15.103:0/2379310498 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd760035030 con 0x55b707275d80 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd766a00640 1 -- 172.21.15.103:0/2379310498 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd760040070 con 0x55b707275d80 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.570+0000 7fd766a00640 1 -- 172.21.15.103:0/2379310498 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7600038e0 con 0x55b707275d80 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.571+0000 7fd77f21b740 1 -- 172.21.15.103:0/2379310498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b707275d80 msgr2=0x55b707278170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.571+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b707275d80 0x55b707278170 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd76002a960 tx=0x7fd760005de0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 -- 172.21.15.103:0/2379310498 shutdown_connections 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 0x55b70727d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2379310498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b707275d80 0x55b707278170 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 -- 172.21.15.103:0/2379310498 >> 172.21.15.103:0/2379310498 conn(0x55b70726cac0 msgr2=0x55b70726d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 -- 172.21.15.103:0/2379310498 shutdown_connections 2024-09-11T00:30:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 -- 172.21.15.103:0/2379310498 wait complete. 2024-09-11T00:30:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 -- start start 2024-09-11T00:30:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.572+0000 7fd77f21b740 1 Processor -- start 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b707275d80 0x55b70727f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b70727fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 0x55b707280710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd77f21b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b70710bab0 con 0x55b7072786b0 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd77f21b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55b70717a570 con 0x55b70727b0f0 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd77f21b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b70727e070 con 0x55b707275d80 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd77f21b740 10 client.0 _ll_register_callbacks cb 0x55b707189100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd779000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b707275d80 0x55b70727f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd779000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b707275d80 0x55b70727f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44658/0 (socket says 172.21.15.103:44658) 2024-09-11T00:30:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd779000640 1 -- 172.21.15.103:0/4101013085 learned_addr learned my addr 172.21.15.103:0/4101013085 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd779000640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 0x55b707280710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.573+0000 7fd779a00640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b70727fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd779a00640 1 -- 172.21.15.103:0/4101013085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b707275d80 msgr2=0x55b70727f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd779a00640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b707275d80 0x55b70727f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd779a00640 1 -- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 msgr2=0x55b707280710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd779a00640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 0x55b707280710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd779a00640 1 -- 172.21.15.103:0/4101013085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b706f35790 con 0x55b7072786b0 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd779000640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b707275d80 0x55b70727f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd779a00640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b70727fbb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd768012700 tx=0x7fd768012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca665ab8e21ddc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd75b400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd75b400640 1 -- 172.21.15.103:0/4101013085 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768004050 con 0x55b7072786b0 2024-09-11T00:30:20.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd760040070 con 0x55b7072786b0 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.574+0000 7fd75b400640 1 -- 172.21.15.103:0/4101013085 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fd7680041f0 con 0x55b7072786b0 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd75b400640 1 -- 172.21.15.103:0/4101013085 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd768025050 con 0x55b7072786b0 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 10 client.16077 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b706fae620 con 0x55b7072786b0 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd75b400640 1 -- 172.21.15.103:0/4101013085 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd760040070 con 0x55b7072786b0 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd75b400640 1 -- 172.21.15.103:0/4101013085 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd768003900 con 0x55b7072786b0 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 10 client.16077 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 20 client.16077 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b70728ae50 0x55b70728d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 4 client.16077 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.575+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7071e6600 con 0x55b70728ae50 2024-09-11T00:30:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.576+0000 7fd779000640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b70728ae50 0x55b70728d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.576+0000 7fd779000640 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b70728ae50 0x55b70728d240 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b83fd7f9eea774 server_cookie=2435e9fe8e538782 in_seq=0 out_seq=0 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.576+0000 7fd75b400640 10 client.16077 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd75b400640 1 -- 172.21.15.103:0/4101013085 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7071e6600 con 0x55b70728ae50 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd75b400640 10 client.16077 handle_command_reply: tid=0 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 client.16077 shutdown 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 msgr2=0x55b70727fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b70727fbb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd768012700 tx=0x7fd768012bc0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 shutdown_connections 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b70727b0f0 0x55b707280710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b70727fbb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b70728ae50 0x55b70728d240 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 --2- 172.21.15.103:0/4101013085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b707275d80 0x55b70727f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 >> 172.21.15.103:0/4101013085 conn(0x55b70726cac0 msgr2=0x55b70726bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd75b400640 0 client.16077 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 shutdown_connections 2024-09-11T00:30:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 1 -- 172.21.15.103:0/4101013085 wait complete. 2024-09-11T00:30:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.577+0000 7fd77f21b740 20 client.16077 trim_cache size 0 max 16384 2024-09-11T00:30:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 Processor -- start 2024-09-11T00:30:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 -- start start 2024-09-11T00:30:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b70727bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b70727b0f0 0x55b70727fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7741abba0 con 0x55b70727b0f0 2024-09-11T00:30:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd7741a9470 con 0x55b707275d80 2024-09-11T00:30:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.596+0000 7fd77f21b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7741a72c0 con 0x55b7072786b0 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44674/0 (socket says 172.21.15.103:44674) 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 -- 172.21.15.103:0/2899550294 learned_addr learned my addr 172.21.15.103:0/2899550294 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b70727bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779000640 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b70727b0f0 0x55b70727fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 -- 172.21.15.103:0/2899550294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 msgr2=0x55b70727bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b70727bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 -- 172.21.15.103:0/2899550294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b70727b0f0 msgr2=0x55b70727fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b70727b0f0 0x55b70727fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 -- 172.21.15.103:0/2899550294 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7072784e0 con 0x55b7072786b0 2024-09-11T00:30:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.597+0000 7fd779a00640 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727f510 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd768010640 tx=0x7fd768010670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1673d3af42ce57ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.598+0000 7fd766000640 1 -- 172.21.15.103:0/2899550294 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76802d020 con 0x55b7072786b0 2024-09-11T00:30:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.598+0000 7fd766000640 1 -- 172.21.15.103:0/2899550294 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd768003aa0 con 0x55b7072786b0 2024-09-11T00:30:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.598+0000 7fd766000640 1 -- 172.21.15.103:0/2899550294 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7680050c0 con 0x55b7072786b0 2024-09-11T00:30:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.598+0000 7fd77f21b740 1 -- 172.21.15.103:0/2899550294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 msgr2=0x55b70727f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.598+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727f510 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd768010640 tx=0x7fd768010670 comp rx=0 tx=0).stop 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 -- 172.21.15.103:0/2899550294 shutdown_connections 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b70727b0f0 0x55b70727fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7072786b0 0x55b70727f510 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 --2- 172.21.15.103:0/2899550294 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b70727bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 -- 172.21.15.103:0/2899550294 >> 172.21.15.103:0/2899550294 conn(0x55b70726cac0 msgr2=0x55b70727b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 -- 172.21.15.103:0/2899550294 shutdown_connections 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 -- 172.21.15.103:0/2899550294 wait complete. 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 -- start start 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 Processor -- start 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.599+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b7072871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b707286020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd77f21b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b70727b0f0 0x55b707286710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd77f21b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7741a7930 con 0x55b7072786b0 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd77f21b740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd7741a8da0 con 0x55b707275d80 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd77f21b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd7741a8000 con 0x55b70727b0f0 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd77f21b740 10 client.0 _ll_register_callbacks cb 0x7fd7680097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b707286020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b707286020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:57366/0 (socket says 172.21.15.103:57366) 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 -- 172.21.15.103:0/1450452630 learned_addr learned my addr 172.21.15.103:0/1450452630 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779000640 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b7072871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779000640 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b70727b0f0 0x55b707286710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 -- 172.21.15.103:0/1450452630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b70727b0f0 msgr2=0x55b707286710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b70727b0f0 0x55b707286710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 -- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 msgr2=0x55b7072871e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b7072871e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 -- 172.21.15.103:0/1450452630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b707284a50 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd779a00640 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b707286020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd768010010 tx=0x7fd768009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cbcb9bca1a1991 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.600+0000 7fd75b400640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd75b400640 1 -- 172.21.15.103:0/1450452630 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76802d030 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd75b400640 1 -- 172.21.15.103:0/1450452630 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fd768022040 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd768003aa0 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd75b400640 1 -- 172.21.15.103:0/1450452630 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd768040050 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 10 client.16083 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b7072f8080 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd75b400640 1 -- 172.21.15.103:0/1450452630 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd768003aa0 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd75b400640 1 -- 172.21.15.103:0/1450452630 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd768010710 con 0x55b7072786b0 2024-09-11T00:30:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 10 client.16083 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 20 client.16083 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b7072f89d0 0x55b7072fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 4 client.16083 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.601+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7071e20b0 con 0x55b7072f89d0 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.602+0000 7fd779000640 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b7072f89d0 0x55b7072fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.602+0000 7fd75b400640 10 client.16083 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.602+0000 7fd779000640 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b7072f89d0 0x55b7072fadf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1139e7c59c5762e9 server_cookie=9bebe908ca736315 in_seq=0 out_seq=0 2024-09-11T00:30:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.617+0000 7fd75b400640 1 -- 172.21.15.103:0/1450452630 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7071e20b0 con 0x55b7072f89d0 2024-09-11T00:30:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.617+0000 7fd75b400640 10 client.16083 handle_command_reply: tid=0 2024-09-11T00:30:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.617+0000 7fd77f21b740 1 client.16083 shutdown 2024-09-11T00:30:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 msgr2=0x55b707286020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b707286020 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd768010010 tx=0x7fd768009e80 comp rx=0 tx=0).stop 2024-09-11T00:30:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 shutdown_connections 2024-09-11T00:30:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55b7072f89d0 0x55b7072fadf0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b70727b0f0 0x55b707286710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b7072786b0 0x55b707286020 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd75b400640 0 client.16083 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 --2- 172.21.15.103:0/1450452630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55b707275d80 0x55b7072871e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 >> 172.21.15.103:0/1450452630 conn(0x55b70726cac0 msgr2=0x55b70727f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 shutdown_connections 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 1 -- 172.21.15.103:0/1450452630 wait complete. 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.618+0000 7fd77f21b740 20 client.16083 trim_cache size 0 max 16384 2024-09-11T00:30:20.620 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:30:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:30:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:30:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:30:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd75c0b52e0 msgr2=0x7fd75c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd75c0b52e0 0x7fd75c0b77a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0a8ec0 tx=0x7fd76c000f40 comp rx=0 tx=0).stop 2024-09-11T00:30:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 msgr2=0x7fd774199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 0x7fd774199e30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00eb10 tx=0x7fd76c00c2d0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 shutdown_connections 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd75c0b52e0 0x7fd75c0b77a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7741042a0 0x7fd774199e30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd7741038e0 0x7fd774199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 --2- 172.21.15.103:0/3681600605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd774102790 0x7fd774198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 >> 172.21.15.103:0/3681600605 conn(0x7fd7740fe030 msgr2=0x7fd774106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 shutdown_connections 2024-09-11T00:30:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:20.619+0000 7fd77ae00640 1 -- 172.21.15.103:0/3681600605 wait complete. 2024-09-11T00:30:20.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:20 smithi103 ceph-mon[30578]: pgmap v617: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 462 op/s 2024-09-11T00:30:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:20 smithi117 ceph-mon[35525]: pgmap v617: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 462 op/s 2024-09-11T00:30:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:20 smithi183 ceph-mon[35934]: pgmap v617: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 462 op/s 2024-09-11T00:30:22.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:22 smithi103 ceph-mon[30578]: pgmap v618: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 474 op/s 2024-09-11T00:30:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:22 smithi117 ceph-mon[35525]: pgmap v618: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 474 op/s 2024-09-11T00:30:22.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:22 smithi183 ceph-mon[35934]: pgmap v618: 129 pgs: 129 active+clean; 78 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.2 MiB/s wr, 474 op/s 2024-09-11T00:30:24.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:24 smithi103 ceph-mon[30578]: pgmap v619: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 433 op/s 2024-09-11T00:30:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:24 smithi117 ceph-mon[35525]: pgmap v619: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 433 op/s 2024-09-11T00:30:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:24 smithi183 ceph-mon[35934]: pgmap v619: 129 pgs: 129 active+clean; 80 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.1 MiB/s wr, 433 op/s 2024-09-11T00:30:24.899 INFO:tasks.workunit.client.0.smithi103.stdout:truncating to largest ever: 0x63e9b52 2024-09-11T00:30:26.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:26 smithi103 ceph-mon[30578]: pgmap v620: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 517 op/s 2024-09-11T00:30:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:26 smithi117 ceph-mon[35525]: pgmap v620: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 517 op/s 2024-09-11T00:30:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:26 smithi183 ceph-mon[35934]: pgmap v620: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 517 op/s 2024-09-11T00:30:28.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:28 smithi103 ceph-mon[30578]: pgmap v621: 129 pgs: 129 active+clean; 92 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1019 KiB/s wr, 375 op/s 2024-09-11T00:30:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:28 smithi117 ceph-mon[35525]: pgmap v621: 129 pgs: 129 active+clean; 92 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1019 KiB/s wr, 375 op/s 2024-09-11T00:30:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:28 smithi183 ceph-mon[35934]: pgmap v621: 129 pgs: 129 active+clean; 92 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1019 KiB/s wr, 375 op/s 2024-09-11T00:30:29.439 INFO:tasks.workunit.client.0.smithi103.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-09-11T00:30:30.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:30 smithi103 ceph-mon[30578]: pgmap v622: 129 pgs: 129 active+clean; 83 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 450 op/s 2024-09-11T00:30:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:30 smithi117 ceph-mon[35525]: pgmap v622: 129 pgs: 129 active+clean; 83 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 450 op/s 2024-09-11T00:30:30.755 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-09-11T00:30:30.755 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:30:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:30 smithi183 ceph-mon[35934]: pgmap v622: 129 pgs: 129 active+clean; 83 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.1 MiB/s wr, 450 op/s 2024-09-11T00:30:31.247 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.920+0000 7f5795600640 1 -- 172.21.15.103:0/1203214725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010a9d0 msgr2=0x7f579010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.920+0000 7f5795600640 1 --2- 172.21.15.103:0/1203214725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010a9d0 0x7f579010ae10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f577c009900 tx=0x7f577c02a410 comp rx=0 tx=0).stop 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.921+0000 7f5795600640 1 -- 172.21.15.103:0/1203214725 shutdown_connections 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.921+0000 7f5795600640 1 --2- 172.21.15.103:0/1203214725 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f579010b350 0x7f5790111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.921+0000 7f5795600640 1 --2- 172.21.15.103:0/1203214725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010a9d0 0x7f579010ae10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.921+0000 7f5795600640 1 --2- 172.21.15.103:0/1203214725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5790104fc0 0x7f57901053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.921+0000 7f5795600640 1 -- 172.21.15.103:0/1203214725 >> 172.21.15.103:0/1203214725 conn(0x7f57901009f0 msgr2=0x7f5790102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.921+0000 7f5795600640 1 -- 172.21.15.103:0/1203214725 shutdown_connections 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.921+0000 7f5795600640 1 -- 172.21.15.103:0/1203214725 wait complete. 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 Processor -- start 2024-09-11T00:30:31.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 -- start start 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5790104fc0 0x7f579019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579010a9d0 0x7f579019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010b350 0x7f57901a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5790114fa0 con 0x7f5790104fc0 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5790114e20 con 0x7f579010a9d0 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f5795600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5790115120 con 0x7f579010b350 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f578e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579010a9d0 0x7f579019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f578ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5790104fc0 0x7f579019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.922+0000 7f578e000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579010a9d0 0x7f579019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:39500/0 (socket says 172.21.15.103:39500) 2024-09-11T00:30:31.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578e000640 1 -- 172.21.15.103:0/3621983154 learned_addr learned my addr 172.21.15.103:0/3621983154 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578f400640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010b350 0x7f57901a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578e000640 1 -- 172.21.15.103:0/3621983154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010b350 msgr2=0x7f57901a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578e000640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010b350 0x7f57901a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578e000640 1 -- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5790104fc0 msgr2=0x7f579019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578e000640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5790104fc0 0x7f579019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578e000640 1 -- 172.21.15.103:0/3621983154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57901a14e0 con 0x7f579010a9d0 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.923+0000 7f578e000640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579010a9d0 0x7f579019b970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f577c005ab0 tx=0x7f577c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.924+0000 7f5787400640 1 -- 172.21.15.103:0/3621983154 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c03f070 con 0x7f579010a9d0 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.924+0000 7f5787400640 1 -- 172.21.15.103:0/3621983154 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c031b60 con 0x7f579010a9d0 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.924+0000 7f5787400640 1 -- 172.21.15.103:0/3621983154 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c035030 con 0x7f579010a9d0 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.924+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57901a1770 con 0x7f579010a9d0 2024-09-11T00:30:31.927 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.924+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57901a7e90 con 0x7f579010a9d0 2024-09-11T00:30:31.928 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.925+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57901053a0 con 0x7f579010a9d0 2024-09-11T00:30:31.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.928+0000 7f5787400640 1 -- 172.21.15.103:0/3621983154 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f577c03b460 con 0x7f579010a9d0 2024-09-11T00:30:31.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.929+0000 7f5787400640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57740b5360 0x7f57740b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:31.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.929+0000 7f5787400640 1 -- 172.21.15.103:0/3621983154 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f577c106ed0 con 0x7f579010a9d0 2024-09-11T00:30:31.931 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.929+0000 7f578ea00640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57740b5360 0x7f57740b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:31.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.930+0000 7f578ea00640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57740b5360 0x7f57740b7820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f57800095f0 tx=0x7f5780005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:31.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:31.932+0000 7f5787400640 1 -- 172.21.15.103:0/3621983154 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f577c049030 con 0x7f579010a9d0 2024-09-11T00:30:32.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.231+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57901a8160 con 0x7f579010a9d0 2024-09-11T00:30:32.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.233+0000 7f5787400640 1 -- 172.21.15.103:0/3621983154 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11933 (secure 0 0 0) 0x7f577c0e2e20 con 0x7f579010a9d0 2024-09-11T00:30:32.236 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:30:32.236 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 43 2024-09-11T00:30:32.236 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":43,"btime":"2024-09-11T00:29:39:196325+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:29:39.092328+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25858":{"gid":25858,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2940154743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2940154743},{"type":"v1","addr":"172.21.15.183:6835","nonce":2940154743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57740b5360 msgr2=0x7f57740b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57740b5360 0x7f57740b7820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f57800095f0 tx=0x7f5780005f90 comp rx=0 tx=0).stop 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579010a9d0 msgr2=0x7f579019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579010a9d0 0x7f579019b970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f577c005ab0 tx=0x7f577c003c10 comp rx=0 tx=0).stop 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 shutdown_connections 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f57740b5360 0x7f57740b7820 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f579010b350 0x7f57901a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f579010a9d0 0x7f579019b970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 --2- 172.21.15.103:0/3621983154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5790104fc0 0x7f579019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 >> 172.21.15.103:0/3621983154 conn(0x7f57901009f0 msgr2=0x7f5790102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.237+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 shutdown_connections 2024-09-11T00:30:32.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:32.238+0000 7f5795600640 1 -- 172.21.15.103:0/3621983154 wait complete. 2024-09-11T00:30:32.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:32 smithi103 ceph-mon[30578]: pgmap v623: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 451 op/s 2024-09-11T00:30:32.659 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:32 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3621983154' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:32 smithi117 ceph-mon[35525]: pgmap v623: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 451 op/s 2024-09-11T00:30:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:32 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3621983154' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:32 smithi183 ceph-mon[35934]: pgmap v623: 129 pgs: 129 active+clean; 84 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 451 op/s 2024-09-11T00:30:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:32 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3621983154' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:33.256 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0' 2024-09-11T00:30:33.256 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0 2024-09-11T00:30:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:30:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.763+0000 7f0983400640 1 -- 172.21.15.103:0/3622498827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c105ac0 msgr2=0x7f097c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.763+0000 7f0983400640 1 --2- 172.21.15.103:0/3622498827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c105ac0 0x7f097c107f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0968009900 tx=0x7f096802a420 comp rx=0 tx=0).stop 2024-09-11T00:30:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.764+0000 7f0983400640 1 -- 172.21.15.103:0/3622498827 shutdown_connections 2024-09-11T00:30:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.764+0000 7f0983400640 1 --2- 172.21.15.103:0/3622498827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f097c108670 0x7f097c10ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.764+0000 7f0983400640 1 --2- 172.21.15.103:0/3622498827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c105ac0 0x7f097c107f30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.764+0000 7f0983400640 1 --2- 172.21.15.103:0/3622498827 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097c06b470 0x7f097c105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.764+0000 7f0983400640 1 -- 172.21.15.103:0/3622498827 >> 172.21.15.103:0/3622498827 conn(0x7f097c0fd140 msgr2=0x7f097c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.764+0000 7f0983400640 1 -- 172.21.15.103:0/3622498827 shutdown_connections 2024-09-11T00:30:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.764+0000 7f0983400640 1 -- 172.21.15.103:0/3622498827 wait complete. 2024-09-11T00:30:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.765+0000 7f0983400640 1 Processor -- start 2024-09-11T00:30:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.765+0000 7f0983400640 1 -- start start 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0983400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097c06b470 0x7f097c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0983400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f097c105ac0 0x7f097c1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0983400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c108670 0x7f097c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0983400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f097c10e690 con 0x7f097c06b470 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0983400640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f097c10e510 con 0x7f097c108670 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0983400640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f097c10e810 con 0x7f097c105ac0 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0981400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c108670 0x7f097c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0981400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c108670 0x7f097c19a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40926/0 (socket says 172.21.15.103:40926) 2024-09-11T00:30:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.766+0000 7f0981400640 1 -- 172.21.15.103:0/1654446560 learned_addr learned my addr 172.21.15.103:0/1654446560 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0981400640 1 -- 172.21.15.103:0/1654446560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f097c105ac0 msgr2=0x7f097c1993d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T00:30:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0981400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f097c105ac0 0x7f097c1993d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0981400640 1 -- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097c06b470 msgr2=0x7f097c198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0981400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097c06b470 0x7f097c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0981400640 1 -- 172.21.15.103:0/1654446560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c19a8c0 con 0x7f097c108670 2024-09-11T00:30:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0981400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c108670 0x7f097c19a000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f097400b990 tx=0x7f097400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0979600640 1 -- 172.21.15.103:0/1654446560 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974019070 con 0x7f097c108670 2024-09-11T00:30:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0979600640 1 -- 172.21.15.103:0/1654446560 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f097400cba0 con 0x7f097c108670 2024-09-11T00:30:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0979600640 1 -- 172.21.15.103:0/1654446560 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09740075d0 con 0x7f097c108670 2024-09-11T00:30:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.767+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c19abb0 con 0x7f097c108670 2024-09-11T00:30:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.768+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c101570 con 0x7f097c108670 2024-09-11T00:30:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.771+0000 7f09863e7740 1 Processor -- start 2024-09-11T00:30:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.771+0000 7f0979600640 1 -- 172.21.15.103:0/1654446560 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0974014440 con 0x7f097c108670 2024-09-11T00:30:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.771+0000 7f09863e7740 1 -- start start 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f09863e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f09863e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f09863e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f09863e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f27d3146b0 con 0x55f27d3a6b00 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f09863e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f27d303ae0 con 0x55f27d3a1790 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f09863e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f27d3078f0 con 0x55f27d3a40c0 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f097be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f097be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40936/0 (socket says 172.21.15.103:40936) 2024-09-11T00:30:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f097be00640 1 -- 172.21.15.103:0/4139055905 learned_addr learned my addr 172.21.15.103:0/4139055905 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f0980a00640 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.772+0000 7f097be00640 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f097be00640 1 -- 172.21.15.103:0/4139055905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 msgr2=0x55f27d3a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f097be00640 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f097be00640 1 -- 172.21.15.103:0/4139055905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 msgr2=0x55f27d3a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f097be00640 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f097be00640 1 -- 172.21.15.103:0/4139055905 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27d2d0780 con 0x55f27d3a1790 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f097be00640 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3a3b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0968031e10 tx=0x7f09680314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=350936f50bc9fb85 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f096dc00640 1 -- 172.21.15.103:0/4139055905 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968044070 con 0x55f27d3a1790 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f096dc00640 1 -- 172.21.15.103:0/4139055905 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096803cb10 con 0x55f27d3a1790 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.773+0000 7f096dc00640 1 -- 172.21.15.103:0/4139055905 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968035030 con 0x55f27d3a1790 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f0979600640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f09640b53a0 0x7f09640b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f0979600640 1 -- 172.21.15.103:0/1654446560 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f09740e5790 con 0x7f097c108670 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f0980a00640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f09640b53a0 0x7f09640b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 -- 172.21.15.103:0/4139055905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 msgr2=0x55f27d3a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3a3b80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0968031e10 tx=0x7f09680314a0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 -- 172.21.15.103:0/4139055905 shutdown_connections 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 --2- 172.21.15.103:0/4139055905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3a3b80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 -- 172.21.15.103:0/4139055905 >> 172.21.15.103:0/4139055905 conn(0x55f27d39c8e0 msgr2=0x55f27d39d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 -- 172.21.15.103:0/4139055905 shutdown_connections 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.774+0000 7f09863e7740 1 -- 172.21.15.103:0/4139055905 wait complete. 2024-09-11T00:30:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.775+0000 7f0980a00640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f09640b53a0 0x7f09640b7860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f097000aca0 tx=0x7f097000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.775+0000 7f09863e7740 1 -- start start 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.775+0000 7f09863e7740 1 Processor -- start 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f09863e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f09863e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f09863e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f09863e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f27d23bab0 con 0x55f27d3a6b00 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f09863e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f27d2aa570 con 0x55f27d3a1790 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f09863e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f27d3a9a80 con 0x55f27d3a40c0 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f09863e7740 10 client.0 _ll_register_callbacks cb 0x55f27d2b9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0980a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0981400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0980a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3aae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40946/0 (socket says 172.21.15.103:40946) 2024-09-11T00:30:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0980a00640 1 -- 172.21.15.103:0/3416849833 learned_addr learned my addr 172.21.15.103:0/3416849833 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f097be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0981400640 1 -- 172.21.15.103:0/3416849833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 msgr2=0x55f27d3ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0981400640 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0981400640 1 -- 172.21.15.103:0/3416849833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 msgr2=0x55f27d3aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0981400640 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f0981400640 1 -- 172.21.15.103:0/3416849833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27d065790 con 0x55f27d3a6b00 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.776+0000 7f097be00640 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f0981400640 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3ac0d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f09740ab860 tx=0x7f09740ab730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b7c01da67008a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f0962000640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f0962000640 1 -- 172.21.15.103:0/3416849833 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09740abb60 con 0x55f27d3a6b00 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f0962000640 1 -- 172.21.15.103:0/3416849833 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f0974014760 con 0x55f27d3a6b00 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096803cb10 con 0x55f27d3a6b00 2024-09-11T00:30:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f0962000640 1 -- 172.21.15.103:0/3416849833 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0974014960 con 0x55f27d3a6b00 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f09863e7740 10 client.16095 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.777+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f27d0de620 con 0x55f27d3a6b00 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f0962000640 1 -- 172.21.15.103:0/3416849833 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f096803cb10 con 0x55f27d3a6b00 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f0962000640 1 -- 172.21.15.103:0/3416849833 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0974094050 con 0x55f27d3a6b00 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f09863e7740 10 client.16095 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f09863e7740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f09863e7740 20 client.16095 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f09863e7740 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d3b66e0 0x55f27d3b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f09863e7740 4 client.16095 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.778+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f27d316600 con 0x55f27d3b66e0 2024-09-11T00:30:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.779+0000 7f0980a00640 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d3b66e0 0x55f27d3b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.779+0000 7f0962000640 10 client.16095 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.779+0000 7f0980a00640 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d3b66e0 0x55f27d3b8b70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53adb8e695d6c28f server_cookie=e94d1e2fe6c10b52 in_seq=0 out_seq=0 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.780+0000 7f0962000640 1 -- 172.21.15.103:0/3416849833 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f27d316600 con 0x55f27d3b66e0 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.780+0000 7f0962000640 10 client.16095 handle_command_reply: tid=0 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.780+0000 7f09863e7740 1 client.16095 shutdown 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.780+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 msgr2=0x55f27d3ac0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.780+0000 7f09863e7740 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3ac0d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f09740ab860 tx=0x7f09740ab730 comp rx=0 tx=0).stop 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.780+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 shutdown_connections 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a6b00 0x55f27d3ac0d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d3b66e0 0x55f27d3b8b70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 1 --2- 172.21.15.103:0/3416849833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f0962000640 0 client.16095 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 >> 172.21.15.103:0/3416849833 conn(0x55f27d39c8e0 msgr2=0x55f27d39ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 shutdown_connections 2024-09-11T00:30:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 1 -- 172.21.15.103:0/3416849833 wait complete. 2024-09-11T00:30:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.781+0000 7f09863e7740 20 client.16095 trim_cache size 0 max 16384 2024-09-11T00:30:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.798+0000 7f09863e7740 1 Processor -- start 2024-09-11T00:30:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.798+0000 7f09863e7740 1 -- start start 2024-09-11T00:30:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f09863e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a1790 0x55f27d397960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f09863e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f09863e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a6b00 0x55f27d3a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f09863e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1b4490 con 0x55f27d3a1790 2024-09-11T00:30:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f09863e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1b1d60 con 0x55f27d3a6b00 2024-09-11T00:30:33.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f09863e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1afbb0 con 0x55f27d3a40c0 2024-09-11T00:30:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:54552/0 (socket says 172.21.15.103:54552) 2024-09-11T00:30:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 -- 172.21.15.103:0/3483069548 learned_addr learned my addr 172.21.15.103:0/3483069548 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f0980a00640 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a1790 0x55f27d397960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.803 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f0981400640 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a6b00 0x55f27d3a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 -- 172.21.15.103:0/3483069548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a6b00 msgr2=0x55f27d3a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a6b00 0x55f27d3a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 -- 172.21.15.103:0/3483069548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a1790 msgr2=0x55f27d397960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a1790 0x55f27d397960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.799+0000 7f097be00640 1 -- 172.21.15.103:0/3483069548 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27d3a3ef0 con 0x55f27d3a40c0 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.800+0000 7f097be00640 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a6ee0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0968031e10 tx=0x7f096803d3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6578c66622a85e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.800+0000 7f096d200640 1 -- 172.21.15.103:0/3483069548 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096804d070 con 0x55f27d3a40c0 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.800+0000 7f096d200640 1 -- 172.21.15.103:0/3483069548 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09680054f0 con 0x55f27d3a40c0 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.800+0000 7f096d200640 1 -- 172.21.15.103:0/3483069548 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968035040 con 0x55f27d3a40c0 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 -- 172.21.15.103:0/3483069548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 msgr2=0x55f27d3a6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a6ee0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0968031e10 tx=0x7f096803d3f0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 -- 172.21.15.103:0/3483069548 shutdown_connections 2024-09-11T00:30:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a6b00 0x55f27d3a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a40c0 0x55f27d3a6ee0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 --2- 172.21.15.103:0/3483069548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a1790 0x55f27d397960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 -- 172.21.15.103:0/3483069548 >> 172.21.15.103:0/3483069548 conn(0x55f27d39c8e0 msgr2=0x55f27d3a0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 -- 172.21.15.103:0/3483069548 shutdown_connections 2024-09-11T00:30:33.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.801+0000 7f09863e7740 1 -- 172.21.15.103:0/3483069548 wait complete. 2024-09-11T00:30:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 -- start start 2024-09-11T00:30:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 Processor -- start 2024-09-11T00:30:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a40c0 0x55f27d3b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a6b00 0x55f27d3b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1b0220 con 0x55f27d3a40c0 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1b1690 con 0x55f27d3a1790 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f097c1b08f0 con 0x55f27d3a6b00 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.802+0000 7f09863e7740 10 client.0 _ll_register_callbacks cb 0x7f09740a81b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40960/0 (socket says 172.21.15.103:40960) 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 -- 172.21.15.103:0/4197393366 learned_addr learned my addr 172.21.15.103:0/4197393366 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0981400640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a6b00 0x55f27d3b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f097be00640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a40c0 0x55f27d3b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 -- 172.21.15.103:0/4197393366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a6b00 msgr2=0x55f27d3b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a6b00 0x55f27d3b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 -- 172.21.15.103:0/4197393366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a40c0 msgr2=0x55f27d3b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a40c0 0x55f27d3b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 -- 172.21.15.103:0/4197393366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27d3b0440 con 0x55f27d3a1790 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f097be00640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a40c0 0x55f27d3b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0980a00640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3b2bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55f27d3b93f0 tx=0x7f09700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f34743e1ccd95f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.803+0000 7f0962000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.804+0000 7f0962000640 1 -- 172.21.15.103:0/4197393366 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970016070 con 0x55f27d3a1790 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.804+0000 7f0962000640 1 -- 172.21.15.103:0/4197393366 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f0970005020 con 0x55f27d3a1790 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.804+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09680054f0 con 0x55f27d3a1790 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.804+0000 7f0962000640 1 -- 172.21.15.103:0/4197393366 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0970039050 con 0x55f27d3a1790 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.804+0000 7f09863e7740 10 client.26165 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.804+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f27d423a30 con 0x55f27d3a1790 2024-09-11T00:30:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.804+0000 7f0962000640 1 -- 172.21.15.103:0/4197393366 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09680054f0 con 0x55f27d3a1790 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f0962000640 1 -- 172.21.15.103:0/4197393366 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f097002e660 con 0x55f27d3a1790 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f09863e7740 10 client.26165 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f09863e7740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f09863e7740 20 client.26165 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f09863e7740 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d424360 0x55f27d426750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f09863e7740 4 client.26165 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f27d3120b0 con 0x55f27d424360 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.805+0000 7f097be00640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d424360 0x55f27d426750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.806+0000 7f0962000640 10 client.26165 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.806+0000 7f097be00640 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d424360 0x55f27d426750 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b172f2a31d664f53 server_cookie=6b31ab4be7051e2b in_seq=0 out_seq=0 2024-09-11T00:30:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f0962000640 1 -- 172.21.15.103:0/4197393366 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f27d3120b0 con 0x55f27d424360 2024-09-11T00:30:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f0962000640 10 client.26165 handle_command_reply: tid=0 2024-09-11T00:30:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 client.26165 shutdown 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 msgr2=0x55f27d3b2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3b2bb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55f27d3b93f0 tx=0x7f09700046c0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 shutdown_connections 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f27d424360 0x55f27d426750 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f27d3a6b00 0x55f27d3b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f27d3a40c0 0x55f27d3b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f0962000640 0 client.26165 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 --2- 172.21.15.103:0/4197393366 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f27d3a1790 0x55f27d3b2bb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 >> 172.21.15.103:0/4197393366 conn(0x55f27d39c8e0 msgr2=0x55f27d3a0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 shutdown_connections 2024-09-11T00:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 1 -- 172.21.15.103:0/4197393366 wait complete. 2024-09-11T00:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.834+0000 7f09863e7740 20 client.26165 trim_cache size 0 max 16384 2024-09-11T00:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:30:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:30:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.835+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f09640b53a0 msgr2=0x7f09640b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.835+0000 7f0983400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f09640b53a0 0x7f09640b7860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f097000aca0 tx=0x7f097000a2d0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.835+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c108670 msgr2=0x7f097c19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.835+0000 7f0983400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c108670 0x7f097c19a000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f097400b990 tx=0x7f097400be50 comp rx=0 tx=0).stop 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 shutdown_connections 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f09640b53a0 0x7f09640b7860 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f097c108670 0x7f097c19a000 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f097c105ac0 0x7f097c1993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 --2- 172.21.15.103:0/1654446560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f097c06b470 0x7f097c198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 >> 172.21.15.103:0/1654446560 conn(0x7f097c0fd140 msgr2=0x7f097c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 shutdown_connections 2024-09-11T00:30:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:33.836+0000 7f0983400640 1 -- 172.21.15.103:0/1654446560 wait complete. 2024-09-11T00:30:33.855 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0x63ed6b9 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.431+0000 7fc81d200640 1 -- 172.21.15.103:0/3074650298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 msgr2=0x7fc818103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.431+0000 7fc81d200640 1 --2- 172.21.15.103:0/3074650298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 0x7fc818103170 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc808009900 tx=0x7fc80802a400 comp rx=0 tx=0).stop 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.432+0000 7fc81d200640 1 -- 172.21.15.103:0/3074650298 shutdown_connections 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.432+0000 7fc81d200640 1 --2- 172.21.15.103:0/3074650298 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc818109120 0x7fc81810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.432+0000 7fc81d200640 1 --2- 172.21.15.103:0/3074650298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8181087a0 0x7fc818108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.432+0000 7fc81d200640 1 --2- 172.21.15.103:0/3074650298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 0x7fc818103170 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.432+0000 7fc81d200640 1 -- 172.21.15.103:0/3074650298 >> 172.21.15.103:0/3074650298 conn(0x7fc8180fe820 msgr2=0x7fc818100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:34.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.432+0000 7fc81d200640 1 -- 172.21.15.103:0/3074650298 shutdown_connections 2024-09-11T00:30:34.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.432+0000 7fc81d200640 1 -- 172.21.15.103:0/3074650298 wait complete. 2024-09-11T00:30:34.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.433+0000 7fc81d200640 1 Processor -- start 2024-09-11T00:30:34.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.433+0000 7fc81d200640 1 -- start start 2024-09-11T00:30:34.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.433+0000 7fc81d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 0x7fc818199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.433+0000 7fc81d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8181087a0 0x7fc818199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.433+0000 7fc81d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc818109120 0x7fc81819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc81d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc818112de0 con 0x7fc8181087a0 2024-09-11T00:30:34.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc81d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc818112c60 con 0x7fc818109120 2024-09-11T00:30:34.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc81d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc818112f60 con 0x7fc818102d90 2024-09-11T00:30:34.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc815e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 0x7fc818199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc815e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 0x7fc818199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:54564/0 (socket says 172.21.15.103:54564) 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc815e00640 1 -- 172.21.15.103:0/2754049613 learned_addr learned my addr 172.21.15.103:0/2754049613 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc816800640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc818109120 0x7fc81819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc815400640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8181087a0 0x7fc818199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc816800640 1 -- 172.21.15.103:0/2754049613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 msgr2=0x7fc818199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc816800640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 0x7fc818199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc816800640 1 -- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8181087a0 msgr2=0x7fc818199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc816800640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8181087a0 0x7fc818199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.434+0000 7fc816800640 1 -- 172.21.15.103:0/2754049613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81819f400 con 0x7fc818109120 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.435+0000 7fc816800640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc818109120 0x7fc81819ec30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc810009fd0 tx=0x7fc81000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.435+0000 7fc80ea00640 1 -- 172.21.15.103:0/2754049613 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81000ce20 con 0x7fc818109120 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.435+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81819f6f0 con 0x7fc818109120 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.435+0000 7fc80ea00640 1 -- 172.21.15.103:0/2754049613 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc810005020 con 0x7fc818109120 2024-09-11T00:30:34.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.435+0000 7fc80ea00640 1 -- 172.21.15.103:0/2754049613 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810010690 con 0x7fc818109120 2024-09-11T00:30:34.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.435+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc818075550 con 0x7fc818109120 2024-09-11T00:30:34.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.439+0000 7fc81fbd6740 1 Processor -- start 2024-09-11T00:30:34.442 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.440+0000 7fc80ea00640 1 -- 172.21.15.103:0/2754049613 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc8100054d0 con 0x7fc818109120 2024-09-11T00:30:34.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.440+0000 7fc81fbd6740 1 -- start start 2024-09-11T00:30:34.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34df630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607d343dca0 con 0x5607d34c8180 2024-09-11T00:30:34.443 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607d342d0d0 con 0x5607d34dd240 2024-09-11T00:30:34.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607d3430ee0 con 0x5607d34c8aa0 2024-09-11T00:30:34.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc815400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc815400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39294/0 (socket says 172.21.15.103:39294) 2024-09-11T00:30:34.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.441+0000 7fc815400640 1 -- 172.21.15.103:0/1241489139 learned_addr learned my addr 172.21.15.103:0/1241489139 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:34.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.442+0000 7fc815400640 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.444 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.442+0000 7fc815400640 1 -- 172.21.15.103:0/1241489139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 msgr2=0x5607d34dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:34.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.442+0000 7fc815400640 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.442+0000 7fc815400640 1 -- 172.21.15.103:0/1241489139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 msgr2=0x5607d34df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:34.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.442+0000 7fc815400640 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34df630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.442+0000 7fc815400640 1 -- 172.21.15.103:0/1241489139 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d33df400 con 0x5607d34c8180 2024-09-11T00:30:34.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.443+0000 7fc80ea00640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7fc0b52e0 0x7fc7fc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.443+0000 7fc815400640 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34c8560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc8040127a0 tx=0x7fc804009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5e7509dd33bcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:34.445 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.443+0000 7fc80ea00640 1 -- 172.21.15.103:0/2754049613 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc8100e6690 con 0x7fc818109120 2024-09-11T00:30:34.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.444+0000 7fc815e00640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7fc0b52e0 0x7fc7fc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.444+0000 7fc803400640 1 -- 172.21.15.103:0/1241489139 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804012d20 con 0x5607d34c8180 2024-09-11T00:30:34.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.444+0000 7fc803400640 1 -- 172.21.15.103:0/1241489139 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc804010c80 con 0x5607d34c8180 2024-09-11T00:30:34.446 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.444+0000 7fc803400640 1 -- 172.21.15.103:0/1241489139 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80401ebe0 con 0x5607d34c8180 2024-09-11T00:30:34.447 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.445+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1241489139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 msgr2=0x5607d34c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:34.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.445+0000 7fc815e00640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7fc0b52e0 0x7fc7fc0b77a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc808031440 tx=0x7fc808005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:34.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.445+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34c8560 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc8040127a0 tx=0x7fc804009700 comp rx=0 tx=0).stop 2024-09-11T00:30:34.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1241489139 shutdown_connections 2024-09-11T00:30:34.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1241489139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34c8560 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.448 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1241489139 >> 172.21.15.103:0/1241489139 conn(0x5607d34c5fb0 msgr2=0x5607d34c7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:34.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1241489139 shutdown_connections 2024-09-11T00:30:34.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1241489139 wait complete. 2024-09-11T00:30:34.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.446+0000 7fc81fbd6740 1 -- start start 2024-09-11T00:30:34.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.447+0000 7fc81fbd6740 1 Processor -- start 2024-09-11T00:30:34.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.447+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34dcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.449 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.447+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34da020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.447+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.447+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5607d328e7a0 con 0x5607d34c8180 2024-09-11T00:30:34.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.447+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5607d33d2a10 con 0x5607d34dd240 2024-09-11T00:30:34.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.447+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607d34dfec0 con 0x5607d34c8aa0 2024-09-11T00:30:34.450 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.448+0000 7fc81fbd6740 10 client.0 _ll_register_callbacks cb 0x5607d33e2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:34.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.448+0000 7fc815e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34dcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.448+0000 7fc815e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34dcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39298/0 (socket says 172.21.15.103:39298) 2024-09-11T00:30:34.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.448+0000 7fc816800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.448+0000 7fc815e00640 1 -- 172.21.15.103:0/1211332926 learned_addr learned my addr 172.21.15.103:0/1211332926 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:34.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.448+0000 7fc815400640 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34da020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.451 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc815400640 1 -- 172.21.15.103:0/1211332926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 msgr2=0x5607d34da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc815400640 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc815400640 1 -- 172.21.15.103:0/1211332926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 msgr2=0x5607d34dcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc815400640 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34dcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc815400640 1 -- 172.21.15.103:0/1211332926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d319ab90 con 0x5607d34c8aa0 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc815e00640 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc815400640 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34da020 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc8040127a0 tx=0x7fc80401e880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16e84c3b7e62462 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.449+0000 7fc7f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.450+0000 7fc7f7e00640 1 -- 172.21.15.103:0/1211332926 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8040109e0 con 0x5607d34c8aa0 2024-09-11T00:30:34.452 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.450+0000 7fc7f7e00640 1 -- 172.21.15.103:0/1211332926 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc804004020 con 0x5607d34c8aa0 2024-09-11T00:30:34.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.450+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc804010c80 con 0x5607d34c8aa0 2024-09-11T00:30:34.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.450+0000 7fc7f7e00640 1 -- 172.21.15.103:0/1211332926 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc8040042a0 con 0x5607d34c8aa0 2024-09-11T00:30:34.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.450+0000 7fc81fbd6740 10 client.26191 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:34.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.450+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5607d34eb890 con 0x5607d34c8aa0 2024-09-11T00:30:34.453 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc7f7e00640 1 -- 172.21.15.103:0/1211332926 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc804010c80 con 0x5607d34c8aa0 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc7f7e00640 1 -- 172.21.15.103:0/1211332926 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc804032020 con 0x5607d34c8aa0 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc81fbd6740 10 client.26191 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc81fbd6740 10 client.26191 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc81fbd6740 20 client.26191 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d34ecd30 0x5607d34ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc81fbd6740 4 client.26191 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.451+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607d343fbf0 con 0x5607d34ecd30 2024-09-11T00:30:34.454 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.452+0000 7fc815e00640 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d34ecd30 0x5607d34ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.452+0000 7fc7f7e00640 10 client.26191 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:34.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.452+0000 7fc815e00640 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d34ecd30 0x5607d34ef120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15547a0ae4df3fb6 server_cookie=724f85c5bc72babc in_seq=0 out_seq=0 2024-09-11T00:30:34.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.453+0000 7fc7f7e00640 1 -- 172.21.15.103:0/1211332926 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607d343fbf0 con 0x5607d34ecd30 2024-09-11T00:30:34.455 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.453+0000 7fc7f7e00640 10 client.26191 handle_command_reply: tid=0 2024-09-11T00:30:34.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.453+0000 7fc81fbd6740 1 client.26191 shutdown 2024-09-11T00:30:34.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 msgr2=0x5607d34da020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:34.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34da020 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc8040127a0 tx=0x7fc80401e880 comp rx=0 tx=0).stop 2024-09-11T00:30:34.456 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 shutdown_connections 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34dd240 0x5607d34da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d34ecd30 0x5607d34ef120 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34c8aa0 0x5607d34da020 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/1211332926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34dcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc7f7e00640 0 client.26191 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 >> 172.21.15.103:0/1211332926 conn(0x5607d34c5fb0 msgr2=0x5607d34cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 shutdown_connections 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.454+0000 7fc81fbd6740 1 -- 172.21.15.103:0/1211332926 wait complete. 2024-09-11T00:30:34.457 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.455+0000 7fc81fbd6740 20 client.26191 trim_cache size 0 max 16384 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.478+0000 7fc81fbd6740 1 Processor -- start 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc81fbd6740 1 -- start start 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34c8180 0x5607d34e90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8aa0 0x5607d34d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34f4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8181b0200 con 0x5607d34c8aa0 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8181adad0 con 0x5607d34c8180 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8181ab920 con 0x5607d34dd240 2024-09-11T00:30:34.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.479+0000 7fc816800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc816800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34f4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:54594/0 (socket says 172.21.15.103:54594) 2024-09-11T00:30:34.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc816800640 1 -- 172.21.15.103:0/2796131967 learned_addr learned my addr 172.21.15.103:0/2796131967 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:34.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc815400640 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8aa0 0x5607d34d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc815400640 1 -- 172.21.15.103:0/2796131967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 msgr2=0x5607d34f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:34.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc815400640 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc815400640 1 -- 172.21.15.103:0/2796131967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34c8180 msgr2=0x5607d34e90f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc815400640 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34c8180 0x5607d34e90f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.480+0000 7fc815400640 1 -- 172.21.15.103:0/2796131967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607d34c76c0 con 0x5607d34c8aa0 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.481+0000 7fc815400640 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8aa0 0x5607d34d98b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc80401e6f0 tx=0x7fc804009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d2692a0ea6521 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.481+0000 7fc802a00640 1 -- 172.21.15.103:0/2796131967 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804005640 con 0x5607d34c8aa0 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.481+0000 7fc802a00640 1 -- 172.21.15.103:0/2796131967 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc804009a30 con 0x5607d34c8aa0 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.481+0000 7fc802a00640 1 -- 172.21.15.103:0/2796131967 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80401f420 con 0x5607d34c8aa0 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.482+0000 7fc81fbd6740 1 -- 172.21.15.103:0/2796131967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8aa0 msgr2=0x5607d34d98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:34.484 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.482+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8aa0 0x5607d34d98b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc80401e6f0 tx=0x7fc804009590 comp rx=0 tx=0).stop 2024-09-11T00:30:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.482+0000 7fc81fbd6740 1 -- 172.21.15.103:0/2796131967 shutdown_connections 2024-09-11T00:30:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.482+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.482+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8aa0 0x5607d34d98b0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc80401e6f0 tx=0x7fc804009590 comp rx=0 tx=0).stop 2024-09-11T00:30:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.482+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/2796131967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34c8180 0x5607d34e90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.482+0000 7fc81fbd6740 1 -- 172.21.15.103:0/2796131967 >> 172.21.15.103:0/2796131967 conn(0x5607d34c5fb0 msgr2=0x5607d34df0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.483+0000 7fc81fbd6740 1 -- 172.21.15.103:0/2796131967 shutdown_connections 2024-09-11T00:30:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.484+0000 7fc81fbd6740 1 -- 172.21.15.103:0/2796131967 wait complete. 2024-09-11T00:30:34.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.484+0000 7fc81fbd6740 1 -- start start 2024-09-11T00:30:34.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.484+0000 7fc81fbd6740 1 Processor -- start 2024-09-11T00:30:34.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.485+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34e5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.485+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34e2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.485+0000 7fc81fbd6740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34e2660 0x5607d34e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.487 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.485+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8181abf90 con 0x5607d34c8180 2024-09-11T00:30:34.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.485+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8181ad400 con 0x5607d34e2660 2024-09-11T00:30:34.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.485+0000 7fc81fbd6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8181ac660 con 0x5607d34dd240 2024-09-11T00:30:34.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.485+0000 7fc81fbd6740 10 client.0 _ll_register_callbacks cb 0x7fc804009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:34.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34e5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc816800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34e2660 0x5607d34e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34e5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:39314/0 (socket says 172.21.15.103:39314) 2024-09-11T00:30:34.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 -- 172.21.15.103:0/3297739800 learned_addr learned my addr 172.21.15.103:0/3297739800 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:34.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc816800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34e2660 0x5607d34e2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41004/0 (socket says 172.21.15.103:41004) 2024-09-11T00:30:34.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815400640 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34e2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 -- 172.21.15.103:0/3297739800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 msgr2=0x5607d34e2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34e2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 -- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34e2660 msgr2=0x5607d34e2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34e2660 0x5607d34e2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 -- 172.21.15.103:0/3297739800 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607d3559960 con 0x5607d34c8180 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc816800640 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34e2660 0x5607d34e2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc815e00640 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34e5680 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc808004770 tx=0x7fc8080058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dfebb3d057fcb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.486+0000 7fc7f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.487+0000 7fc7f7e00640 1 -- 172.21.15.103:0/3297739800 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80804e070 con 0x5607d34c8180 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.487+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5607d343b6a0 con 0x5607d34c8180 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.487+0000 7fc7f7e00640 1 -- 172.21.15.103:0/3297739800 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc808004410 con 0x5607d34c8180 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.487+0000 7fc7f7e00640 1 -- 172.21.15.103:0/3297739800 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc808035040 con 0x5607d34c8180 2024-09-11T00:30:34.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.487+0000 7fc81fbd6740 10 client.16119 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.487+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5607d355aac0 con 0x5607d34c8180 2024-09-11T00:30:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc7f7e00640 1 -- 172.21.15.103:0/3297739800 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5607d343b6a0 con 0x5607d34c8180 2024-09-11T00:30:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc7f7e00640 1 -- 172.21.15.103:0/3297739800 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc80803c6b0 con 0x5607d34c8180 2024-09-11T00:30:34.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc81fbd6740 10 client.16119 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc81fbd6740 10 client.16119 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc81fbd6740 20 client.16119 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d355b2b0 0x5607d355d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc81fbd6740 4 client.16119 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}] 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.488+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}) -- 0x5607d3438ad0 con 0x5607d355b2b0 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.489+0000 7fc815400640 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d355b2b0 0x5607d355d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.489+0000 7fc815400640 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d355b2b0 0x5607d355d6d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fc05917db31cda1 server_cookie=7bfe4955f709fc4a in_seq=0 out_seq=0 2024-09-11T00:30:34.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:34.489+0000 7fc7f7e00640 10 client.16119 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:34.674 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:34 smithi103 ceph-mon[30578]: pgmap v624: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.0 MiB/s wr, 423 op/s 2024-09-11T00:30:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:34 smithi117 ceph-mon[35525]: pgmap v624: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.0 MiB/s wr, 423 op/s 2024-09-11T00:30:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:34 smithi183 ceph-mon[35934]: pgmap v624: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.0 MiB/s wr, 423 op/s 2024-09-11T00:30:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.432+0000 7fc7f7e00640 1 -- 172.21.15.103:0/3297739800 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5607d3438ad0 con 0x5607d355b2b0 2024-09-11T00:30:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.432+0000 7fc7f7e00640 10 client.16119 handle_command_reply: tid=0 2024-09-11T00:30:35.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.433+0000 7fc81fbd6740 1 client.16119 shutdown 2024-09-11T00:30:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 msgr2=0x5607d34e5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34e5680 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc808004770 tx=0x7fc8080058d0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 shutdown_connections 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5607d34e2660 0x5607d34e2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5607d355b2b0 0x5607d355d6d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5607d34dd240 0x5607d34e2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 --2- 172.21.15.103:0/3297739800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5607d34c8180 0x5607d34e5680 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 >> 172.21.15.103:0/3297739800 conn(0x5607d34c5fb0 msgr2=0x5607d34db650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.434+0000 7fc7f7e00640 0 client.16119 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.435+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 shutdown_connections 2024-09-11T00:30:35.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.435+0000 7fc81fbd6740 1 -- 172.21.15.103:0/3297739800 wait complete. 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.435+0000 7fc81fbd6740 20 client.16119 trim_cache size 0 max 16384 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 43, 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 22, 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "56264023": { 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 22, 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:30:35.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 146.8, 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:30:35.439 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:30:35.440 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:30:35.440 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:30:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7fc0b52e0 msgr2=0x7fc7fc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7fc0b52e0 0x7fc7fc0b77a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc808031440 tx=0x7fc808005e50 comp rx=0 tx=0).stop 2024-09-11T00:30:35.440 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc818109120 msgr2=0x7fc81819ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc818109120 0x7fc81819ec30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc810009fd0 tx=0x7fc81000ef10 comp rx=0 tx=0).stop 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 shutdown_connections 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc7fc0b52e0 0x7fc7fc0b77a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc818109120 0x7fc81819ec30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8181087a0 0x7fc818199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 --2- 172.21.15.103:0/2754049613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc818102d90 0x7fc818199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 >> 172.21.15.103:0/2754049613 conn(0x7fc8180fe820 msgr2=0x7fc8181000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 shutdown_connections 2024-09-11T00:30:35.441 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:35.437+0000 7fc81d200640 1 -- 172.21.15.103:0/2754049613 wait complete. 2024-09-11T00:30:36.435 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '56264023', quiesce took 5.7 seconds. Will release after: 21.1 2024-09-11T00:30:36.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:36 smithi103 ceph-mon[30578]: pgmap v625: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 483 op/s 2024-09-11T00:30:36.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:36 smithi117 ceph-mon[35525]: pgmap v625: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 483 op/s 2024-09-11T00:30:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:36 smithi183 ceph-mon[35934]: pgmap v625: 129 pgs: 129 active+clean; 78 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 483 op/s 2024-09-11T00:30:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:38 smithi103 ceph-mon[30578]: pgmap v626: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1020 KiB/s wr, 384 op/s 2024-09-11T00:30:38.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:38 smithi117 ceph-mon[35525]: pgmap v626: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1020 KiB/s wr, 384 op/s 2024-09-11T00:30:38.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:38 smithi183 ceph-mon[35934]: pgmap v626: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1020 KiB/s wr, 384 op/s 2024-09-11T00:30:40.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:40 smithi103 ceph-mon[30578]: pgmap v627: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 407 op/s 2024-09-11T00:30:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:40 smithi117 ceph-mon[35525]: pgmap v627: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 407 op/s 2024-09-11T00:30:40.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:40 smithi183 ceph-mon[35934]: pgmap v627: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.1 MiB/s wr, 407 op/s 2024-09-11T00:30:42.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:42 smithi103 ceph-mon[30578]: pgmap v628: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 921 KiB/s wr, 328 op/s 2024-09-11T00:30:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:42 smithi117 ceph-mon[35525]: pgmap v628: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 921 KiB/s wr, 328 op/s 2024-09-11T00:30:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:42 smithi183 ceph-mon[35934]: pgmap v628: 129 pgs: 129 active+clean; 99 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 921 KiB/s wr, 328 op/s 2024-09-11T00:30:44.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:44 smithi103 ceph-mon[30578]: pgmap v629: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 713 KiB/s wr, 259 op/s 2024-09-11T00:30:44.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:44 smithi117 ceph-mon[35525]: pgmap v629: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 713 KiB/s wr, 259 op/s 2024-09-11T00:30:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:44 smithi183 ceph-mon[35934]: pgmap v629: 129 pgs: 129 active+clean; 101 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 713 KiB/s wr, 259 op/s 2024-09-11T00:30:45.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:45 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:30:45.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:45 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:30:45.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:45 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:30:46.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:46 smithi103 ceph-mon[30578]: pgmap v630: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 615 KiB/s wr, 206 op/s 2024-09-11T00:30:46.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:46 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:30:46.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:46 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:30:46.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:46 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:30:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:46 smithi117 ceph-mon[35525]: pgmap v630: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 615 KiB/s wr, 206 op/s 2024-09-11T00:30:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:46 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:30:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:46 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:30:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:46 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:30:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:46 smithi183 ceph-mon[35934]: pgmap v630: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 615 KiB/s wr, 206 op/s 2024-09-11T00:30:46.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:46 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:30:46.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:46 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:30:46.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:46 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 -- 172.21.15.103:0/1333626172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9768102790 msgr2=0x7f9768102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 --2- 172.21.15.103:0/1333626172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9768102790 0x7f9768102b90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f975c009930 tx=0x7f975c02a420 comp rx=0 tx=0).stop 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 -- 172.21.15.103:0/1333626172 shutdown_connections 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 --2- 172.21.15.103:0/1333626172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97681042a0 0x7f976810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 --2- 172.21.15.103:0/1333626172 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97681038e0 0x7f9768103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 --2- 172.21.15.103:0/1333626172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9768102790 0x7f9768102b90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 -- 172.21.15.103:0/1333626172 >> 172.21.15.103:0/1333626172 conn(0x7f97680fe030 msgr2=0x7f9768100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 -- 172.21.15.103:0/1333626172 shutdown_connections 2024-09-11T00:30:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.956+0000 7f976e200640 1 -- 172.21.15.103:0/1333626172 wait complete. 2024-09-11T00:30:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.957+0000 7f976e200640 1 Processor -- start 2024-09-11T00:30:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.957+0000 7f976e200640 1 -- start start 2024-09-11T00:30:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976e200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9768102790 0x7f976819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976e200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97681038e0 0x7f976819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976e200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97681042a0 0x7f976819c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976e200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f976810e450 con 0x7f97681042a0 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976e200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f976810e2d0 con 0x7f9768102790 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976e200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f976810e5d0 con 0x7f97681038e0 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97681042a0 0x7f976819c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97681042a0 0x7f976819c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60416/0 (socket says 172.21.15.103:60416) 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976d800640 1 -- 172.21.15.103:0/102454237 learned_addr learned my addr 172.21.15.103:0/102454237 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976ce00640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9768102790 0x7f976819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f9767e00640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97681038e0 0x7f976819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976d800640 1 -- 172.21.15.103:0/102454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97681038e0 msgr2=0x7f976819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.958+0000 7f976d800640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97681038e0 0x7f976819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f976d800640 1 -- 172.21.15.103:0/102454237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9768102790 msgr2=0x7f976819ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f976d800640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9768102790 0x7f976819ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f976d800640 1 -- 172.21.15.103:0/102454237 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976819c920 con 0x7f97681042a0 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f976d800640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97681042a0 0x7f976819c060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f976000eb10 tx=0x7f976000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f9765600640 1 -- 172.21.15.103:0/102454237 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760004b20 con 0x7f97681042a0 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f9765600640 1 -- 172.21.15.103:0/102454237 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9760004cc0 con 0x7f97681042a0 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976819cc10 con 0x7f97681042a0 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f9765600640 1 -- 172.21.15.103:0/102454237 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760010670 con 0x7f97681042a0 2024-09-11T00:30:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.959+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a32e0 con 0x7f97681042a0 2024-09-11T00:30:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.963+0000 7f9772693740 1 Processor -- start 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9765600640 1 -- 172.21.15.103:0/102454237 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9760005020 con 0x7f97681042a0 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9772693740 1 -- start start 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9772693740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9772693740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9772693740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9772693740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f88d8206b0 con 0x55f88d8b02a0 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9772693740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f88d80fae0 con 0x55f88d8ad970 2024-09-11T00:30:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9772693740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f88d8138f0 con 0x55f88d8b2ce0 2024-09-11T00:30:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f976ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9767e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9767e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:56012/0 (socket says 172.21.15.103:56012) 2024-09-11T00:30:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9767e00640 1 -- 172.21.15.103:0/3215152326 learned_addr learned my addr 172.21.15.103:0/3215152326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.964+0000 7f9767e00640 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f976ce00640 1 -- 172.21.15.103:0/3215152326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 msgr2=0x55f88d8b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f976ce00640 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f976ce00640 1 -- 172.21.15.103:0/3215152326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 msgr2=0x55f88d8afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f976ce00640 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f976ce00640 1 -- 172.21.15.103:0/3215152326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88d7dc780 con 0x55f88d8b02a0 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f976ce00640 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8b2690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f975c02a940 tx=0x7f975c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce5dee0fb81fc85 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f975a000640 1 -- 172.21.15.103:0/3215152326 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c035030 con 0x55f88d8b02a0 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f975a000640 1 -- 172.21.15.103:0/3215152326 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c040070 con 0x55f88d8b02a0 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f975a000640 1 -- 172.21.15.103:0/3215152326 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c031910 con 0x55f88d8b02a0 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.965+0000 7f9765600640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97500b52e0 0x7f97500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9765600640 1 -- 172.21.15.103:0/102454237 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f97600e6430 con 0x7f97681042a0 2024-09-11T00:30:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9767e00640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97500b52e0 0x7f97500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 -- 172.21.15.103:0/3215152326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 msgr2=0x55f88d8b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8b2690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f975c02a940 tx=0x7f975c005c00 comp rx=0 tx=0).stop 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 -- 172.21.15.103:0/3215152326 shutdown_connections 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8b2690 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 --2- 172.21.15.103:0/3215152326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 -- 172.21.15.103:0/3215152326 >> 172.21.15.103:0/3215152326 conn(0x55f88d8a8ac0 msgr2=0x55f88d8a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 -- 172.21.15.103:0/3215152326 shutdown_connections 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9767e00640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97500b52e0 0x7f97500b77a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9754005bc0 tx=0x7f975400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.966+0000 7f9772693740 1 -- 172.21.15.103:0/3215152326 wait complete. 2024-09-11T00:30:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 -- start start 2024-09-11T00:30:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 Processor -- start 2024-09-11T00:30:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8b7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8b02a0 0x55f88d8b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f88d747ab0 con 0x55f88d8ad970 2024-09-11T00:30:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f88d7b6570 con 0x55f88d8b02a0 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f88d8b5c60 con 0x55f88d8b2ce0 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.967+0000 7f9772693740 10 client.0 _ll_register_callbacks cb 0x55f88d7c5100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f976ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8b02a0 0x55f88d8b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8b02a0 0x55f88d8b76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:38102/0 (socket says 172.21.15.103:38102) 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 -- 172.21.15.103:0/270747528 learned_addr learned my addr 172.21.15.103:0/270747528 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f976d800640 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 -- 172.21.15.103:0/270747528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 msgr2=0x55f88d8b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 -- 172.21.15.103:0/270747528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 msgr2=0x55f88d8b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8b7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 -- 172.21.15.103:0/270747528 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88d571790 con 0x55f88d8b02a0 2024-09-11T00:30:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f976ce00640 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f9767e00640 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8b02a0 0x55f88d8b76b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9754012a40 tx=0x7f9754004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1276d2da243ed59b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f974ea00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.968+0000 7f974ea00640 1 -- 172.21.15.103:0/270747528 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975402e060 con 0x55f88d8b02a0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c040070 con 0x55f88d8b02a0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f974ea00640 1 -- 172.21.15.103:0/270747528 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f975401b050 con 0x55f88d8b02a0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f974ea00640 1 -- 172.21.15.103:0/270747528 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9754035050 con 0x55f88d8b02a0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 10 client.26192 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f88d5ea620 con 0x55f88d8b02a0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f974ea00640 1 -- 172.21.15.103:0/270747528 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c040070 con 0x55f88d8b02a0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f974ea00640 1 -- 172.21.15.103:0/270747528 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f975401d070 con 0x55f88d8b02a0 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 10 client.26192 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 20 client.26192 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8c2950 0x55f88d8c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 4 client.26192 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.969+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f88d822600 con 0x55f88d8c2950 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.970+0000 7f976ce00640 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8c2950 0x55f88d8c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.970+0000 7f974ea00640 10 client.26192 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.970+0000 7f976ce00640 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8c2950 0x55f88d8c4dd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d2e074c5d320eb server_cookie=8b8f427fa672d20d in_seq=0 out_seq=0 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.971+0000 7f974ea00640 1 -- 172.21.15.103:0/270747528 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f88d822600 con 0x55f88d8c2950 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.971+0000 7f974ea00640 10 client.26192 handle_command_reply: tid=0 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.971+0000 7f9772693740 1 client.26192 shutdown 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.971+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8b02a0 msgr2=0x55f88d8b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.971+0000 7f9772693740 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8b02a0 0x55f88d8b76b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9754012a40 tx=0x7f9754004730 comp rx=0 tx=0).stop 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 shutdown_connections 2024-09-11T00:30:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8c2950 0x55f88d8c4dd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b2ce0 0x55f88d8b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8b02a0 0x55f88d8b76b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 --2- 172.21.15.103:0/270747528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f974ea00640 0 client.26192 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 >> 172.21.15.103:0/270747528 conn(0x55f88d8a8ac0 msgr2=0x55f88d8a7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 shutdown_connections 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 1 -- 172.21.15.103:0/270747528 wait complete. 2024-09-11T00:30:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.972+0000 7f9772693740 20 client.26192 trim_cache size 0 max 16384 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 Processor -- start 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 -- start start 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8a8ac0 0x55f88d8b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b02a0 0x55f88d8a9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97681affe0 con 0x55f88d8ad970 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ad8b0 con 0x55f88d8a8ac0 2024-09-11T00:30:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9772693740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ab700 con 0x55f88d8b02a0 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9767e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9767e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8a3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60444/0 (socket says 172.21.15.103:60444) 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f9767e00640 1 -- 172.21.15.103:0/3423769372 learned_addr learned my addr 172.21.15.103:0/3423769372 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f976d800640 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b02a0 0x55f88d8a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.987+0000 7f976ce00640 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8a8ac0 0x55f88d8b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9767e00640 1 -- 172.21.15.103:0/3423769372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b02a0 msgr2=0x55f88d8a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9767e00640 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b02a0 0x55f88d8a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9767e00640 1 -- 172.21.15.103:0/3423769372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8a8ac0 msgr2=0x55f88d8b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9767e00640 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8a8ac0 0x55f88d8b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9767e00640 1 -- 172.21.15.103:0/3423769372 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f88d8b6b10 con 0x55f88d8ad970 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9767e00640 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8a3c20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f975400dea0 tx=0x7f9754010840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52187af2c1ff3e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9759600640 1 -- 172.21.15.103:0/3423769372 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754010ac0 con 0x55f88d8ad970 2024-09-11T00:30:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9759600640 1 -- 172.21.15.103:0/3423769372 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754029920 con 0x55f88d8ad970 2024-09-11T00:30:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.988+0000 7f9759600640 1 -- 172.21.15.103:0/3423769372 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754004a40 con 0x55f88d8ad970 2024-09-11T00:30:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 -- 172.21.15.103:0/3423769372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 msgr2=0x55f88d8a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8a3c20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f975400dea0 tx=0x7f9754010840 comp rx=0 tx=0).stop 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 -- 172.21.15.103:0/3423769372 shutdown_connections 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8b02a0 0x55f88d8a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8ad970 0x55f88d8a3c20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 --2- 172.21.15.103:0/3423769372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8a8ac0 0x55f88d8b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 -- 172.21.15.103:0/3423769372 >> 172.21.15.103:0/3423769372 conn(0x55f88d8b6710 msgr2=0x55f88d8b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 -- 172.21.15.103:0/3423769372 shutdown_connections 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 -- 172.21.15.103:0/3423769372 wait complete. 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 -- start start 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.989+0000 7f9772693740 1 Processor -- start 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9772693740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8a8ac0 0x55f88d8be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9772693740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9772693740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9772693740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97681abd70 con 0x55f88d8b02a0 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9772693740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ad1e0 con 0x55f88d8ad970 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9772693740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97681ac440 con 0x55f88d8a8ac0 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9772693740 10 client.0 _ll_register_callbacks cb 0x7f975400df90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976ce00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8a8ac0 0x55f88d8be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8bcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60446/0 (socket says 172.21.15.103:60446) 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 -- 172.21.15.103:0/1858279790 learned_addr learned my addr 172.21.15.103:0/1858279790 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976ce00640 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8a8ac0 0x55f88d8be050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:56018/0 (socket says 172.21.15.103:56018) 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f9767e00640 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 -- 172.21.15.103:0/1858279790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8a8ac0 msgr2=0x55f88d8be050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8a8ac0 0x55f88d8be050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 -- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 msgr2=0x55f88d8be6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8be6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.990+0000 7f976d800640 1 -- 172.21.15.103:0/1858279790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f88d8bd130 con 0x55f88d8b02a0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f976d800640 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8bcb70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f97600a9ad0 tx=0x7f97600a87b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdb04858aa3642e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f974ea00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f974ea00640 1 -- 172.21.15.103:0/1858279790 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600ac1b0 con 0x55f88d8b02a0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f974ea00640 1 -- 172.21.15.103:0/1858279790 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f97600b5b60 con 0x55f88d8b02a0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9754029920 con 0x55f88d8b02a0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f974ea00640 1 -- 172.21.15.103:0/1858279790 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f97600abfd0 con 0x55f88d8b02a0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 10 client.16137 fetch_fsmap learned FSMap version 43 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f88d930240 con 0x55f88d8b02a0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f974ea00640 1 -- 172.21.15.103:0/1858279790 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754029920 con 0x55f88d8b02a0 2024-09-11T00:30:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f974ea00640 1 -- 172.21.15.103:0/1858279790 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f97600cfbb0 con 0x55f88d8b02a0 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 10 client.16137 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 20 client.16137 populate_metadata read hostname 'smithi103' 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8b2ce0 0x55f88d932be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 4 client.16137 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.991+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f88d81e0b0 con 0x55f88d8b2ce0 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.992+0000 7f976ce00640 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8b2ce0 0x55f88d932be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.992+0000 7f976ce00640 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8b2ce0 0x55f88d932be0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=486edda82b7a5b75 server_cookie=e2381a2d19a05230 in_seq=0 out_seq=0 2024-09-11T00:30:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:46.992+0000 7f974ea00640 10 client.16137 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.008+0000 7f974ea00640 1 -- 172.21.15.103:0/1858279790 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f88d81e0b0 con 0x55f88d8b2ce0 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.008+0000 7f974ea00640 10 client.16137 handle_command_reply: tid=0 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.008+0000 7f9772693740 1 client.16137 shutdown 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.008+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 msgr2=0x55f88d8bcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.008+0000 7f9772693740 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8bcb70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f97600a9ad0 tx=0x7f97600a87b0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 shutdown_connections 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f88d8b02a0 0x55f88d8bcb70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f88d8ad970 0x55f88d8be6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f88d8b2ce0 0x55f88d932be0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 --2- 172.21.15.103:0/1858279790 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f88d8a8ac0 0x55f88d8be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f974ea00640 0 client.16137 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 >> 172.21.15.103:0/1858279790 conn(0x55f88d8b6710 msgr2=0x55f88d8a9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 shutdown_connections 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 1 -- 172.21.15.103:0/1858279790 wait complete. 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f9772693740 20 client.16137 trim_cache size 0 max 16384 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:30:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.009+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97500b52e0 msgr2=0x7f97500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97500b52e0 0x7f97500b77a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9754005bc0 tx=0x7f975400b410 comp rx=0 tx=0).stop 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97681042a0 msgr2=0x7f976819c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97681042a0 0x7f976819c060 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f976000eb10 tx=0x7f976000c2d0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 shutdown_connections 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f97500b52e0 0x7f97500b77a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97681042a0 0x7f976819c060 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97681038e0 0x7f976819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 --2- 172.21.15.103:0/102454237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9768102790 0x7f976819ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 >> 172.21.15.103:0/102454237 conn(0x7f97680fe030 msgr2=0x7f9768075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 shutdown_connections 2024-09-11T00:30:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:30:47.010+0000 7f976e200640 1 -- 172.21.15.103:0/102454237 wait complete. 2024-09-11T00:30:48.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:48 smithi103 ceph-mon[30578]: pgmap v631: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 307 KiB/s wr, 87 op/s 2024-09-11T00:30:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:48 smithi117 ceph-mon[35525]: pgmap v631: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 307 KiB/s wr, 87 op/s 2024-09-11T00:30:48.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:48 smithi183 ceph-mon[35934]: pgmap v631: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 307 KiB/s wr, 87 op/s 2024-09-11T00:30:50.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:50 smithi103 ceph-mon[30578]: pgmap v632: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 238 KiB/s wr, 72 op/s 2024-09-11T00:30:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:50 smithi117 ceph-mon[35525]: pgmap v632: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 238 KiB/s wr, 72 op/s 2024-09-11T00:30:50.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:50 smithi183 ceph-mon[35934]: pgmap v632: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 238 KiB/s wr, 72 op/s 2024-09-11T00:30:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:52 smithi103 ceph-mon[30578]: pgmap v633: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 98 KiB/s wr, 26 op/s 2024-09-11T00:30:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:52 smithi117 ceph-mon[35525]: pgmap v633: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 98 KiB/s wr, 26 op/s 2024-09-11T00:30:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:52 smithi183 ceph-mon[35934]: pgmap v633: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 98 KiB/s wr, 26 op/s 2024-09-11T00:30:54.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:54 smithi103 ceph-mon[30578]: pgmap v634: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-09-11T00:30:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:54 smithi117 ceph-mon[35525]: pgmap v634: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-09-11T00:30:54.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:54 smithi183 ceph-mon[35934]: pgmap v634: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-09-11T00:30:56.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:56 smithi103 ceph-mon[30578]: pgmap v635: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-09-11T00:30:56.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:56 smithi117 ceph-mon[35525]: pgmap v635: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-09-11T00:30:56.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:56 smithi183 ceph-mon[35934]: pgmap v635: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-09-11T00:30:57.536 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:30:57.920 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:30:58.660 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:58 smithi103 ceph-mon[30578]: pgmap v636: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2024-09-11T00:30:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:58 smithi117 ceph-mon[35525]: pgmap v636: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2024-09-11T00:30:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.770+0000 7fc2ede00640 1 -- 172.21.15.103:0/1844919050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8109150 msgr2=0x7fc2e810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.770+0000 7fc2ede00640 1 --2- 172.21.15.103:0/1844919050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8109150 0x7fc2e810f950 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009ca0 tx=0x7fc2e002a8c0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.771+0000 7fc2ede00640 1 -- 172.21.15.103:0/1844919050 shutdown_connections 2024-09-11T00:30:58.773 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.771+0000 7fc2ede00640 1 --2- 172.21.15.103:0/1844919050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e8109150 0x7fc2e810f950 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.771+0000 7fc2ede00640 1 --2- 172.21.15.103:0/1844919050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e81087d0 0x7fc2e8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.771+0000 7fc2ede00640 1 --2- 172.21.15.103:0/1844919050 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2e8102dc0 0x7fc2e81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.771+0000 7fc2ede00640 1 -- 172.21.15.103:0/1844919050 >> 172.21.15.103:0/1844919050 conn(0x7fc2e80fe830 msgr2=0x7fc2e8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.771+0000 7fc2ede00640 1 -- 172.21.15.103:0/1844919050 shutdown_connections 2024-09-11T00:30:58.774 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.771+0000 7fc2ede00640 1 -- 172.21.15.103:0/1844919050 wait complete. 2024-09-11T00:30:58.775 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 Processor -- start 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 -- start start 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e8102dc0 0x7fc2e819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e81087d0 0x7fc2e819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2e8109150 0x7fc2e81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8112dd0 con 0x7fc2e81087d0 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8112c50 con 0x7fc2e8109150 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2ede00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e8112f50 con 0x7fc2e8102dc0 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2e6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e8102dc0 0x7fc2e819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2e6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e81087d0 0x7fc2e819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2e6a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e8102dc0 0x7fc2e819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:42728/0 (socket says 172.21.15.103:42728) 2024-09-11T00:30:58.776 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.772+0000 7fc2e6a00640 1 -- 172.21.15.103:0/4101194180 learned_addr learned my addr 172.21.15.103:0/4101194180 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e7400640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2e8109150 0x7fc2e81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e6a00640 1 -- 172.21.15.103:0/4101194180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2e8109150 msgr2=0x7fc2e81a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e6a00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2e8109150 0x7fc2e81a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e6a00640 1 -- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e81087d0 msgr2=0x7fc2e819b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e6a00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e81087d0 0x7fc2e819b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e6a00640 1 -- 172.21.15.103:0/4101194180 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e81a1560 con 0x7fc2e8102dc0 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e6000640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e81087d0 0x7fc2e819b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2e6a00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e8102dc0 0x7fc2e819b4b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800d950 tx=0x7fc2d800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2eca00640 1 -- 172.21.15.103:0/4101194180 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8017030 con 0x7fc2e8102dc0 2024-09-11T00:30:58.777 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.773+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e81a1850 con 0x7fc2e8102dc0 2024-09-11T00:30:58.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.774+0000 7fc2eca00640 1 -- 172.21.15.103:0/4101194180 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2d8004020 con 0x7fc2e8102dc0 2024-09-11T00:30:58.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.774+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e81a80e0 con 0x7fc2e8102dc0 2024-09-11T00:30:58.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.774+0000 7fc2eca00640 1 -- 172.21.15.103:0/4101194180 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8005650 con 0x7fc2e8102dc0 2024-09-11T00:30:58.778 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.775+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e81031a0 con 0x7fc2e8102dc0 2024-09-11T00:30:58.779 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.777+0000 7fc2eca00640 1 -- 172.21.15.103:0/4101194180 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc2d8009a30 con 0x7fc2e8102dc0 2024-09-11T00:30:58.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:58 smithi183 ceph-mon[35934]: pgmap v636: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 6.8 KiB/s wr, 8 op/s 2024-09-11T00:30:58.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.779+0000 7fc2eca00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc2cc0b5360 0x7fc2cc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:30:58.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.779+0000 7fc2eca00640 1 -- 172.21.15.103:0/4101194180 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fc2d80e5bd0 con 0x7fc2e8102dc0 2024-09-11T00:30:58.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.779+0000 7fc2e6000640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc2cc0b5360 0x7fc2cc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:30:58.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.780+0000 7fc2e6000640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc2cc0b5360 0x7fc2cc0b7820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8104ff0 tx=0x7fc2d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:30:58.782 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:58.780+0000 7fc2eca00640 1 -- 172.21.15.103:0/4101194180 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc2d80ac720 con 0x7fc2e8102dc0 2024-09-11T00:30:59.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.000+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2e819c5e0 con 0x7fc2e8102dc0 2024-09-11T00:30:59.003 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.001+0000 7fc2eca00640 1 -- 172.21.15.103:0/4101194180 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11933 (secure 0 0 0) 0x7fc2e819c5e0 con 0x7fc2e8102dc0 2024-09-11T00:30:59.004 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:30:59.004 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":43,"btime":"2024-09-11T00:29:39:196325+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:29:39.092328+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25858":{"gid":25858,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2940154743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2940154743},{"type":"v1","addr":"172.21.15.183:6835","nonce":2940154743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:30:59.004 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 43 2024-09-11T00:30:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc2cc0b5360 msgr2=0x7fc2cc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc2cc0b5360 0x7fc2cc0b7820 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8104ff0 tx=0x7fc2d4005f90 comp rx=0 tx=0).stop 2024-09-11T00:30:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e8102dc0 msgr2=0x7fc2e819b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:30:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e8102dc0 0x7fc2e819b4b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800d950 tx=0x7fc2d800de10 comp rx=0 tx=0).stop 2024-09-11T00:30:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 shutdown_connections 2024-09-11T00:30:59.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc2cc0b5360 0x7fc2cc0b7820 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc2e8109150 0x7fc2e81a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc2e81087d0 0x7fc2e819b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 --2- 172.21.15.103:0/4101194180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc2e8102dc0 0x7fc2e819b4b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:30:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 >> 172.21.15.103:0/4101194180 conn(0x7fc2e80fe830 msgr2=0x7fc2e8106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:30:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 shutdown_connections 2024-09-11T00:30:59.007 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:30:59.004+0000 7fc2ede00640 1 -- 172.21.15.103:0/4101194180 wait complete. 2024-09-11T00:30:59.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:30:59 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4101194180' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:30:59 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4101194180' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:59.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:30:59 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4101194180' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:30:59.999 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --set-id 56264023 --release --await' 2024-09-11T00:30:59.999 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --set-id 56264023 --release --await 2024-09-11T00:31:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 -- 172.21.15.103:0/2967820890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b8102790 msgr2=0x7f38b8102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 --2- 172.21.15.103:0/2967820890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b8102790 0x7f38b8102b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009930 tx=0x7f38a802a420 comp rx=0 tx=0).stop 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 -- 172.21.15.103:0/2967820890 shutdown_connections 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 --2- 172.21.15.103:0/2967820890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b81042a0 0x7f38b810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 --2- 172.21.15.103:0/2967820890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b81038e0 0x7f38b8103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 --2- 172.21.15.103:0/2967820890 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b8102790 0x7f38b8102b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 -- 172.21.15.103:0/2967820890 >> 172.21.15.103:0/2967820890 conn(0x7f38b80fe030 msgr2=0x7f38b8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 -- 172.21.15.103:0/2967820890 shutdown_connections 2024-09-11T00:31:00.130 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.127+0000 7f38bca00640 1 -- 172.21.15.103:0/2967820890 wait complete. 2024-09-11T00:31:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.128+0000 7f38bca00640 1 Processor -- start 2024-09-11T00:31:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.128+0000 7f38bca00640 1 -- start start 2024-09-11T00:31:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38bca00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b8102790 0x7f38b8105010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38bca00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b81038e0 0x7f38b8105660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38bca00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b81042a0 0x7f38b8106290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38bca00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810e400 con 0x7f38b81042a0 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38bca00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810e280 con 0x7f38b8102790 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38bca00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38b810e580 con 0x7f38b81038e0 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b81042a0 0x7f38b8106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b81042a0 0x7f38b8106290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51146/0 (socket says 172.21.15.103:51146) 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38b7e00640 1 -- 172.21.15.103:0/3160066070 learned_addr learned my addr 172.21.15.103:0/3160066070 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38b7400640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b8102790 0x7f38b8105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.129+0000 7f38b6a00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b81038e0 0x7f38b8105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38b7e00640 1 -- 172.21.15.103:0/3160066070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b81038e0 msgr2=0x7f38b8105660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b81038e0 0x7f38b8105660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38b7e00640 1 -- 172.21.15.103:0/3160066070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b8102790 msgr2=0x7f38b8105010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b8102790 0x7f38b8105010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38b7e00640 1 -- 172.21.15.103:0/3160066070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b8106b50 con 0x7f38b81042a0 2024-09-11T00:31:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b81042a0 0x7f38b8106290 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38b000efd0 tx=0x7f38b000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38afe00640 1 -- 172.21.15.103:0/3160066070 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0003b70 con 0x7f38b81042a0 2024-09-11T00:31:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38afe00640 1 -- 172.21.15.103:0/3160066070 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b0003d10 con 0x7f38b81042a0 2024-09-11T00:31:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b8106e40 con 0x7f38b81042a0 2024-09-11T00:31:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38afe00640 1 -- 172.21.15.103:0/3160066070 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00108a0 con 0x7f38b81042a0 2024-09-11T00:31:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.130+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b81074e0 con 0x7f38b81042a0 2024-09-11T00:31:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.133+0000 7f38c0dd2740 1 Processor -- start 2024-09-11T00:31:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38afe00640 1 -- 172.21.15.103:0/3160066070 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f38b001f020 con 0x7f38b81042a0 2024-09-11T00:31:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38c0dd2740 1 -- start start 2024-09-11T00:31:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3b91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f21b3246b0 con 0x55f21b3b1970 2024-09-11T00:31:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f21b313ae0 con 0x55f21b3b42a0 2024-09-11T00:31:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f21b3178f0 con 0x55f21b3b6ce0 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38b6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38b6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51152/0 (socket says 172.21.15.103:51152) 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.134+0000 7f38b6a00640 1 -- 172.21.15.103:0/1451024917 learned_addr learned my addr 172.21.15.103:0/1451024917 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b7400640 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b6a00640 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3b91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b6a00640 1 -- 172.21.15.103:0/1451024917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 msgr2=0x55f21b3b91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b6a00640 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3b91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b6a00640 1 -- 172.21.15.103:0/1451024917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 msgr2=0x55f21b3b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b6a00640 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b6a00640 1 -- 172.21.15.103:0/1451024917 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21b2e0780 con 0x55f21b3b1970 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b7400640 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:00.138 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38b6a00640 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3b3d60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009f80 tx=0x7f38a40098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565114bc710dd320 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38afe00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f389c0b5360 0x7f389c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38a3e00640 1 -- 172.21.15.103:0/1451024917 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4004050 con 0x55f21b3b1970 2024-09-11T00:31:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.135+0000 7f38afe00640 1 -- 172.21.15.103:0/3160066070 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f38b00e5fe0 con 0x7f38b81042a0 2024-09-11T00:31:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38a3e00640 1 -- 172.21.15.103:0/1451024917 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a40041f0 con 0x55f21b3b1970 2024-09-11T00:31:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38a3e00640 1 -- 172.21.15.103:0/1451024917 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4003850 con 0x55f21b3b1970 2024-09-11T00:31:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38b7400640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f389c0b5360 0x7f389c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38c0dd2740 1 -- 172.21.15.103:0/1451024917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 msgr2=0x55f21b3b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38b7400640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f389c0b5360 0x7f389c0b7820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f38a8006010 tx=0x7f38a80314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3b3d60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009f80 tx=0x7f38a40098b0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38c0dd2740 1 -- 172.21.15.103:0/1451024917 shutdown_connections 2024-09-11T00:31:00.141 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3b91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/1451024917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3b3d60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.136+0000 7f38c0dd2740 1 -- 172.21.15.103:0/1451024917 >> 172.21.15.103:0/1451024917 conn(0x55f21b3acac0 msgr2=0x55f21b3ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.137+0000 7f38c0dd2740 1 -- 172.21.15.103:0/1451024917 shutdown_connections 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.137+0000 7f38c0dd2740 1 -- 172.21.15.103:0/1451024917 wait complete. 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.137+0000 7f38c0dd2740 1 -- start start 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.137+0000 7f38c0dd2740 1 Processor -- start 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b1970 0x55f21b3bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b42a0 0x55f21b3bb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f21b24bab0 con 0x55f21b3b42a0 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f21b2ba570 con 0x55f21b3b6ce0 2024-09-11T00:31:00.142 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f21b3b9cc0 con 0x55f21b3b1970 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38c0dd2740 10 client.0 _ll_register_callbacks cb 0x55f21b2c9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b42a0 0x55f21b3bb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b42a0 0x55f21b3bb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51160/0 (socket says 172.21.15.103:51160) 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 -- 172.21.15.103:0/3508062912 learned_addr learned my addr 172.21.15.103:0/3508062912 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b7400640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b1970 0x55f21b3bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 -- 172.21.15.103:0/3508062912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b1970 msgr2=0x55f21b3bb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b1970 0x55f21b3bb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 -- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b6ce0 msgr2=0x55f21b3bc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b6a00640 1 -- 172.21.15.103:0/3508062912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21b075790 con 0x55f21b3b42a0 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.138+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38b6a00640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b42a0 0x55f21b3bb710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005890 tx=0x7f38a40056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205fc9b2dfe7499a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:00.143 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:31:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38a0c00640 1 -- 172.21.15.103:0/3508062912 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4005c50 con 0x55f21b3b42a0 2024-09-11T00:31:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a40041f0 con 0x55f21b3b42a0 2024-09-11T00:31:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38a0c00640 1 -- 172.21.15.103:0/3508062912 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f38a401f430 con 0x55f21b3b42a0 2024-09-11T00:31:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38a0c00640 1 -- 172.21.15.103:0/3508062912 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f38a4032050 con 0x55f21b3b42a0 2024-09-11T00:31:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38c0dd2740 10 client.16155 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f21b0ee620 con 0x55f21b3b42a0 2024-09-11T00:31:00.144 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.139+0000 7f38a0c00640 1 -- 172.21.15.103:0/3508062912 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38a40041f0 con 0x55f21b3b42a0 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38a0c00640 1 -- 172.21.15.103:0/3508062912 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f38a402c4d0 con 0x55f21b3b42a0 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38c0dd2740 10 client.16155 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38c0dd2740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38c0dd2740 20 client.16155 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b3c66c0 0x55f21b3c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38c0dd2740 4 client.16155 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f21b326600 con 0x55f21b3c66c0 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.140+0000 7f38b7400640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b3c66c0 0x55f21b3c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.141+0000 7f38b7400640 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b3c66c0 0x55f21b3c8ab0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f6a1ccae1a28a2 server_cookie=483054239907cd68 in_seq=0 out_seq=0 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.141+0000 7f38a0c00640 10 client.16155 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.141+0000 7f38a0c00640 1 -- 172.21.15.103:0/3508062912 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f21b326600 con 0x55f21b3c66c0 2024-09-11T00:31:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.141+0000 7f38a0c00640 10 client.16155 handle_command_reply: tid=0 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 client.16155 shutdown 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b42a0 msgr2=0x55f21b3bb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b42a0 0x55f21b3bb710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f38a4005890 tx=0x7f38a40056f0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 shutdown_connections 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b42a0 0x55f21b3bb710 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b3c66c0 0x55f21b3c8ab0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3508062912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b1970 0x55f21b3bb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38a0c00640 0 client.16155 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 >> 172.21.15.103:0/3508062912 conn(0x55f21b3acac0 msgr2=0x55f21b3abbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 shutdown_connections 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3508062912 wait complete. 2024-09-11T00:31:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.142+0000 7f38c0dd2740 20 client.16155 trim_cache size 0 max 16384 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.172+0000 7f38c0dd2740 1 Processor -- start 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.172+0000 7f38c0dd2740 1 -- start start 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b1970 0x55f21b3afb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b42a0 0x55f21b3b0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81b4310 con 0x55f21b3b6ce0 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81b1be0 con 0x55f21b3b1970 2024-09-11T00:31:00.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81afa30 con 0x55f21b3b42a0 2024-09-11T00:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38b7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51170/0 (socket says 172.21.15.103:51170) 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38b7e00640 1 -- 172.21.15.103:0/3511974430 learned_addr learned my addr 172.21.15.103:0/3511974430 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38b6a00640 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b42a0 0x55f21b3b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.173+0000 7f38b7400640 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b1970 0x55f21b3afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38b7e00640 1 -- 172.21.15.103:0/3511974430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b42a0 msgr2=0x55f21b3b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b42a0 0x55f21b3b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38b7e00640 1 -- 172.21.15.103:0/3511974430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b1970 msgr2=0x55f21b3afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b1970 0x55f21b3afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38b7e00640 1 -- 172.21.15.103:0/3511974430 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f21b3bc650 con 0x55f21b3b6ce0 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38b7e00640 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bb070 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f38b00048e0 tx=0x7f38b00abb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a87266813d3399 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38a3400640 1 -- 172.21.15.103:0/3511974430 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00acaf0 con 0x55f21b3b6ce0 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38a3400640 1 -- 172.21.15.103:0/3511974430 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b00ac3f0 con 0x55f21b3b6ce0 2024-09-11T00:31:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.174+0000 7f38a3400640 1 -- 172.21.15.103:0/3511974430 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00a6d50 con 0x55f21b3b6ce0 2024-09-11T00:31:00.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3511974430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b6ce0 msgr2=0x55f21b3bb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bb070 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f38b00048e0 tx=0x7f38b00abb60 comp rx=0 tx=0).stop 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3511974430 shutdown_connections 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b6ce0 0x55f21b3bb070 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b42a0 0x55f21b3b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/3511974430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b1970 0x55f21b3afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3511974430 >> 172.21.15.103:0/3511974430 conn(0x55f21b3acac0 msgr2=0x55f21b3b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3511974430 shutdown_connections 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.175+0000 7f38c0dd2740 1 -- 172.21.15.103:0/3511974430 wait complete. 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.176+0000 7f38c0dd2740 1 -- start start 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.176+0000 7f38c0dd2740 1 Processor -- start 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3c1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38c0dd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81b00a0 con 0x55f21b3b1970 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81b1510 con 0x55f21b3b42a0 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38c0dd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81b0770 con 0x55f21b3b6ce0 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38c0dd2740 10 client.0 _ll_register_callbacks cb 0x7f38a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3c2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51186/0 (socket says 172.21.15.103:51186) 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 -- 172.21.15.103:0/2453271709 learned_addr learned my addr 172.21.15.103:0/2453271709 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b6a00640 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7e00640 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 -- 172.21.15.103:0/2453271709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 msgr2=0x55f21b3c1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3c1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 -- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 msgr2=0x55f21b3c1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3c1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.177+0000 7f38b7400640 1 -- 172.21.15.103:0/2453271709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f21b3c2300 con 0x55f21b3b1970 2024-09-11T00:31:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38b7400640 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3c2a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f38a802a940 tx=0x7f38a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc1c72798ef8650 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38a0c00640 1 -- 172.21.15.103:0/2453271709 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a805e060 con 0x55f21b3b1970 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38a0c00640 1 -- 172.21.15.103:0/2453271709 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f38a80035f0 con 0x55f21b3b1970 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b00ac3f0 con 0x55f21b3b1970 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38a0c00640 1 -- 172.21.15.103:0/2453271709 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f38a8065050 con 0x55f21b3b1970 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38c0dd2740 10 client.16161 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f21b433570 con 0x55f21b3b1970 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.178+0000 7f38a0c00640 1 -- 172.21.15.103:0/2453271709 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38b00ac3f0 con 0x55f21b3b1970 2024-09-11T00:31:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38a0c00640 1 -- 172.21.15.103:0/2453271709 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f38a8044070 con 0x55f21b3b1970 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38c0dd2740 10 client.16161 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38c0dd2740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38c0dd2740 20 client.16161 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b433f00 0x55f21b436320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38c0dd2740 4 client.16161 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f21b3220b0 con 0x55f21b433f00 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.179+0000 7f38b6a00640 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b433f00 0x55f21b436320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.180+0000 7f38b6a00640 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b433f00 0x55f21b436320 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=677a1bc8e585957d server_cookie=96dadb751992b51a in_seq=0 out_seq=0 2024-09-11T00:31:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.180+0000 7f38a0c00640 10 client.16161 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38a0c00640 1 -- 172.21.15.103:0/2453271709 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f21b3220b0 con 0x55f21b433f00 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38a0c00640 10 client.16161 handle_command_reply: tid=0 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 client.16161 shutdown 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 msgr2=0x55f21b3c2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3c2a00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f38a802a940 tx=0x7f38a8009260 comp rx=0 tx=0).stop 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 shutdown_connections 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f21b433f00 0x55f21b436320 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f21b3b6ce0 0x55f21b3c1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f21b3b42a0 0x55f21b3c1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38a0c00640 0 client.16161 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 --2- 172.21.15.103:0/2453271709 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f21b3b1970 0x55f21b3c2a00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.193+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 >> 172.21.15.103:0/2453271709 conn(0x55f21b3acac0 msgr2=0x55f21b3b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 shutdown_connections 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38c0dd2740 1 -- 172.21.15.103:0/2453271709 wait complete. 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38c0dd2740 20 client.16161 trim_cache size 0 max 16384 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:31:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:31:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f389c0b5360 msgr2=0x7f389c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f389c0b5360 0x7f389c0b7820 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f38a8006010 tx=0x7f38a80314a0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b81042a0 msgr2=0x7f38b8106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b81042a0 0x7f38b8106290 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38b000efd0 tx=0x7f38b000c6a0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 shutdown_connections 2024-09-11T00:31:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f389c0b5360 0x7f389c0b7820 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38b81042a0 0x7f38b8106290 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f38b81038e0 0x7f38b8105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.194+0000 7f38bca00640 1 --2- 172.21.15.103:0/3160066070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38b8102790 0x7f38b8105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.195+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 >> 172.21.15.103:0/3160066070 conn(0x7f38b80fe030 msgr2=0x7f38b8075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.195+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 shutdown_connections 2024-09-11T00:31:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:00.195+0000 7f38bca00640 1 -- 172.21.15.103:0/3160066070 wait complete. 2024-09-11T00:31:00.401 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:31:00.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:00 smithi103 ceph-mon[30578]: pgmap v637: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-11T00:31:00.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:00 smithi117 ceph-mon[35525]: pgmap v637: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-11T00:31:00.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:00 smithi183 ceph-mon[35934]: pgmap v637: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.036+0000 7f6872600640 1 -- 172.21.15.103:0/591603563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 msgr2=0x7f686c1117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 --2- 172.21.15.103:0/591603563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 0x7f686c1117d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6864009ca0 tx=0x7f686402a8b0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 -- 172.21.15.103:0/591603563 shutdown_connections 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 --2- 172.21.15.103:0/591603563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 0x7f686c1117d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 --2- 172.21.15.103:0/591603563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075f40 0x7f686c10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 --2- 172.21.15.103:0/591603563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686c075540 0x7f686c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 -- 172.21.15.103:0/591603563 >> 172.21.15.103:0/591603563 conn(0x7f686c0fe830 msgr2=0x7f686c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 -- 172.21.15.103:0/591603563 shutdown_connections 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.037+0000 7f6872600640 1 -- 172.21.15.103:0/591603563 wait complete. 2024-09-11T00:31:01.063 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 Processor -- start 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 -- start start 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075540 0x7f686c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686c075f40 0x7f686c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 0x7f686c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f686c114fd0 con 0x7f686c10f3e0 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f686c114e50 con 0x7f686c075540 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f6872600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f686c115150 con 0x7f686c075f40 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f686aa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686c075f40 0x7f686c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f686be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 0x7f686c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f686b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075540 0x7f686c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.038+0000 7f686b400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075540 0x7f686c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:54632/0 (socket says 172.21.15.103:54632) 2024-09-11T00:31:01.064 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686b400640 1 -- 172.21.15.103:0/20348375 learned_addr learned my addr 172.21.15.103:0/20348375 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686b400640 1 -- 172.21.15.103:0/20348375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686c075f40 msgr2=0x7f686c19b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686b400640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686c075f40 0x7f686c19b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686b400640 1 -- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 msgr2=0x7f686c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686b400640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 0x7f686c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686b400640 1 -- 172.21.15.103:0/20348375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c1a1510 con 0x7f686c075540 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686be00640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 0x7f686c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f686b400640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075540 0x7f686c19b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e9f0 tx=0x7f685c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.039+0000 7f6863e00640 1 -- 172.21.15.103:0/20348375 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c00cd80 con 0x7f686c075540 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.040+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c1a1800 con 0x7f686c075540 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.040+0000 7f6863e00640 1 -- 172.21.15.103:0/20348375 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f685c0054d0 con 0x7f686c075540 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.040+0000 7f6863e00640 1 -- 172.21.15.103:0/20348375 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c010690 con 0x7f686c075540 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.040+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c1a7ec0 con 0x7f686c075540 2024-09-11T00:31:01.065 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 Processor -- start 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 -- start start 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c803d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c8c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cd98bf7290 con 0x55cd98c7fff0 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cd98be66c0 con 0x55cd98c80910 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f687505e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd98bea4d0 con 0x55cd98c8c8a0 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f686be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c803d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f686be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c803d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51214/0 (socket says 172.21.15.103:51214) 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.043+0000 7f686be00640 1 -- 172.21.15.103:0/10996193 learned_addr learned my addr 172.21.15.103:0/10996193 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f686aa00640 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f686be00640 1 -- 172.21.15.103:0/10996193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 msgr2=0x55cd98c8ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:01.066 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f686be00640 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f686be00640 1 -- 172.21.15.103:0/10996193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 msgr2=0x55cd98c8c360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f686be00640 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c8c360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f686be00640 1 -- 172.21.15.103:0/10996193 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd98b73460 con 0x55cd98c7fff0 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f6863e00640 1 -- 172.21.15.103:0/20348375 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f685c004e90 con 0x7f686c075540 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f686be00640 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c803d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6864035000 tx=0x7f6864005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7713b249838c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f6860c00640 1 -- 172.21.15.103:0/10996193 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864030b40 con 0x55cd98c7fff0 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.044+0000 7f6860c00640 1 -- 172.21.15.103:0/10996193 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864036070 con 0x55cd98c7fff0 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f6860c00640 1 -- 172.21.15.103:0/10996193 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864004730 con 0x55cd98c7fff0 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 -- 172.21.15.103:0/10996193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 msgr2=0x55cd98c803d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c803d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6864035000 tx=0x7f6864005fa0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 -- 172.21.15.103:0/10996193 shutdown_connections 2024-09-11T00:31:01.067 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c8c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 --2- 172.21.15.103:0/10996193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c803d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 -- 172.21.15.103:0/10996193 >> 172.21.15.103:0/10996193 conn(0x55cd98c7de20 msgr2=0x55cd98c7efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 -- 172.21.15.103:0/10996193 shutdown_connections 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.045+0000 7f687505e740 1 -- 172.21.15.103:0/10996193 wait complete. 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.046+0000 7f6863e00640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f68500b51f0 0x7f68500b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.046+0000 7f687505e740 1 -- start start 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.046+0000 7f6863e00640 1 -- 172.21.15.103:0/20348375 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f685c0e63a0 con 0x7f686c075540 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.046+0000 7f686aa00640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f68500b51f0 0x7f68500b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.046+0000 7f687505e740 1 Processor -- start 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f687505e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f687505e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c89de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f687505e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.068 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686aa00640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f68500b51f0 0x7f68500b76b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55cd98c7ffa0 tx=0x7f685800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f687505e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cd98a47bb0 con 0x55cd98c7fff0 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f687505e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55cd98b816c0 con 0x55cd98c80910 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f687505e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cd98c8f540 con 0x55cd98c8c8a0 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f687505e740 10 client.0 _ll_register_callbacks cb 0x55cd98b9c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c898a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51216/0 (socket says 172.21.15.103:51216) 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 -- 172.21.15.103:0/2430553313 learned_addr learned my addr 172.21.15.103:0/2430553313 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686be00640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686aa00640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c89de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 -- 172.21.15.103:0/2430553313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 msgr2=0x55cd98c8a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.069 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 -- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 msgr2=0x55cd98c89de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c89de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686b400640 1 -- 172.21.15.103:0/2430553313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd98944b00 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.047+0000 7f686aa00640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c89de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f686b400640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c898a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f685c009380 tx=0x7f685c0ac5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b76942cbe1bd43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f6855600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f6855600640 1 -- 172.21.15.103:0/2430553313 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0d3020 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f6855600640 1 -- 172.21.15.103:0/2430553313 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f685c0b5870 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6864036070 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f6855600640 1 -- 172.21.15.103:0/2430553313 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f685c0b1020 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f687505e740 10 client.16173 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cd989bb960 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.048+0000 7f6855600640 1 -- 172.21.15.103:0/2430553313 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864036070 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f6855600640 1 -- 172.21.15.103:0/2430553313 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f685c0c3030 con 0x55cd98c7fff0 2024-09-11T00:31:01.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f687505e740 10 client.16173 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f687505e740 10 client.16173 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f687505e740 20 client.16173 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f687505e740 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98c9c450 0x55cd98c9e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f687505e740 4 client.16173 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd98bf91e0 con 0x55cd98c9c450 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.049+0000 7f686aa00640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98c9c450 0x55cd98c9e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.050+0000 7f686aa00640 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98c9c450 0x55cd98c9e880 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4450282bc3bb2bbe server_cookie=f0191c53c0c4f4b3 in_seq=0 out_seq=0 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.050+0000 7f6855600640 10 client.16173 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f6855600640 1 -- 172.21.15.103:0/2430553313 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cd98bf91e0 con 0x55cd98c9c450 2024-09-11T00:31:01.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f6855600640 10 client.16173 handle_command_reply: tid=0 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 client.16173 shutdown 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 msgr2=0x55cd98c898a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c898a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f685c009380 tx=0x7f685c0ac5c0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 shutdown_connections 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98c9c450 0x55cd98c9e880 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c8c8a0 0x55cd98c8a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c80910 0x55cd98c89de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 --2- 172.21.15.103:0/2430553313 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c898a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f6855600640 0 client.16173 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 >> 172.21.15.103:0/2430553313 conn(0x55cd98c7de20 msgr2=0x55cd98c839c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 shutdown_connections 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 1 -- 172.21.15.103:0/2430553313 wait complete. 2024-09-11T00:31:01.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.051+0000 7f687505e740 20 client.16173 trim_cache size 0 max 16384 2024-09-11T00:31:01.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.069+0000 7f687505e740 1 Processor -- start 2024-09-11T00:31:01.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.069+0000 7f687505e740 1 -- start start 2024-09-11T00:31:01.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.069+0000 7f687505e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c7fff0 0x55cd98c88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.069+0000 7f687505e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c80910 0x55cd98c86ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f687505e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f687505e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1b4520 con 0x55cd98c80910 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f687505e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1b1df0 con 0x55cd98c8c8a0 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f687505e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1afc40 con 0x55cd98c7fff0 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c80910 0x55cd98c86ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c80910 0x55cd98c86ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51232/0 (socket says 172.21.15.103:51232) 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 -- 172.21.15.103:0/1343288007 learned_addr learned my addr 172.21.15.103:0/1343288007 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686b400640 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c7fff0 0x55cd98c88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686be00640 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c87010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 -- 172.21.15.103:0/1343288007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c7fff0 msgr2=0x55cd98c88140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c7fff0 0x55cd98c88140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 -- 172.21.15.103:0/1343288007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 msgr2=0x55cd98c87010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c87010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.070+0000 7f686aa00640 1 -- 172.21.15.103:0/1343288007 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd98c7f630 con 0x55cd98c80910 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.071+0000 7f686aa00640 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c80910 0x55cd98c86ad0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6858003d10 tx=0x7f6858016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadc53d4829969e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.071+0000 7f6857e00640 1 -- 172.21.15.103:0/1343288007 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6858003dc0 con 0x55cd98c80910 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.071+0000 7f6857e00640 1 -- 172.21.15.103:0/1343288007 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68580200d0 con 0x55cd98c80910 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.071+0000 7f6857e00640 1 -- 172.21.15.103:0/1343288007 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685801e420 con 0x55cd98c80910 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.071+0000 7f687505e740 1 -- 172.21.15.103:0/1343288007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c80910 msgr2=0x55cd98c86ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.071+0000 7f687505e740 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c80910 0x55cd98c86ad0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6858003d10 tx=0x7f6858016000 comp rx=0 tx=0).stop 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 -- 172.21.15.103:0/1343288007 shutdown_connections 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c80910 0x55cd98c86ad0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 --2- 172.21.15.103:0/1343288007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c7fff0 0x55cd98c88140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 -- 172.21.15.103:0/1343288007 >> 172.21.15.103:0/1343288007 conn(0x55cd98c7de20 msgr2=0x55cd98c7cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:01.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 -- 172.21.15.103:0/1343288007 shutdown_connections 2024-09-11T00:31:01.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 -- 172.21.15.103:0/1343288007 wait complete. 2024-09-11T00:31:01.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 -- start start 2024-09-11T00:31:01.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.072+0000 7f687505e740 1 Processor -- start 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f687505e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c91830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f687505e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c80910 0x55cd98c91d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f687505e740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f687505e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1b02b0 con 0x55cd98c7fff0 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f687505e740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1b1720 con 0x55cd98c8c8a0 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f687505e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f686c1b0980 con 0x55cd98c80910 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f687505e740 10 client.0 _ll_register_callbacks cb 0x7f685c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f686b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c91830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f686b400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c91830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51242/0 (socket says 172.21.15.103:51242) 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f686b400640 1 -- 172.21.15.103:0/1877872331 learned_addr learned my addr 172.21.15.103:0/1877872331 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f686aa00640 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c80910 0x55cd98c91d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.073+0000 7f686be00640 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f686b400640 1 -- 172.21.15.103:0/1877872331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c80910 msgr2=0x55cd98c91d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f686b400640 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c80910 0x55cd98c91d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f686b400640 1 -- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 msgr2=0x55cd98c922b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f686b400640 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c922b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f686b400640 1 -- 172.21.15.103:0/1877872331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd98c88820 con 0x55cd98c7fff0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f686b400640 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c91830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f685c0b1410 tx=0x7f685c0a8540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b02037b36b098 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f6855600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f6855600640 1 -- 172.21.15.103:0/1877872331 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c0d3040 con 0x55cd98c7fff0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f6855600640 1 -- 172.21.15.103:0/1877872331 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f685c0abbd0 con 0x55cd98c7fff0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68580200d0 con 0x55cd98c7fff0 2024-09-11T00:31:01.080 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f6855600640 1 -- 172.21.15.103:0/1877872331 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f685c0a8610 con 0x55cd98c7fff0 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f687505e740 10 client.16179 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.074+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cd98d0a610 con 0x55cd98c7fff0 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f6855600640 1 -- 172.21.15.103:0/1877872331 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68580200d0 con 0x55cd98c7fff0 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f6855600640 1 -- 172.21.15.103:0/1877872331 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f685c0c3070 con 0x55cd98c7fff0 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f687505e740 10 client.16179 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f687505e740 10 client.16179 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f687505e740 20 client.16179 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f687505e740 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98d0adc0 0x55cd98d0d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f687505e740 4 client.16179 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "56264023", "release": true, "await": true}] 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "set_id": "56264023", "release": true, "await": true}) -- 0x55cd98bf4c90 con 0x55cd98d0adc0 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.075+0000 7f686aa00640 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98d0adc0 0x55cd98d0d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.076+0000 7f686aa00640 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98d0adc0 0x55cd98d0d1e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8da3a69bb2e3f0 server_cookie=adbb073becc68abd in_seq=0 out_seq=0 2024-09-11T00:31:01.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.076+0000 7f6855600640 10 client.16179 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:01.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f6855600640 1 -- 172.21.15.103:0/1877872331 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cd98bf4c90 con 0x55cd98d0adc0 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f6855600640 10 client.16179 handle_command_reply: tid=0 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 client.16179 shutdown 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 msgr2=0x55cd98c91830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c91830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f685c0b1410 tx=0x7f685c0a8540 comp rx=0 tx=0).stop 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 shutdown_connections 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55cd98c8c8a0 0x55cd98c922b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55cd98d0adc0 0x55cd98d0d1e0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cd98c80910 0x55cd98c91d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 --2- 172.21.15.103:0/1877872331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cd98c7fff0 0x55cd98c91830 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f6855600640 0 client.16179 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 >> 172.21.15.103:0/1877872331 conn(0x55cd98c7de20 msgr2=0x55cd98c7cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:01.247 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 shutdown_connections 2024-09-11T00:31:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.244+0000 7f687505e740 1 -- 172.21.15.103:0/1877872331 wait complete. 2024-09-11T00:31:01.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.245+0000 7f687505e740 20 client.16179 trim_cache size 0 max 16384 2024-09-11T00:31:01.248 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:31:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 43, 2024-09-11T00:31:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:31:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 24, 2024-09-11T00:31:01.248 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "56264023": { 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 24, 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 146.8, 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:31:01.249 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:01.250 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f68500b51f0 msgr2=0x7f68500b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f68500b51f0 0x7f68500b76b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55cd98c7ffa0 tx=0x7f685800b390 comp rx=0 tx=0).stop 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075540 msgr2=0x7f686c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075540 0x7f686c19b460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f685c00e9f0 tx=0x7f685c00edf0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 shutdown_connections 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f68500b51f0 0x7f68500b76b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f686c10f3e0 0x7f686c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f686c075f40 0x7f686c19b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 --2- 172.21.15.103:0/20348375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f686c075540 0x7f686c19b460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 >> 172.21.15.103:0/20348375 conn(0x7f686c0fe830 msgr2=0x7f686c1115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 shutdown_connections 2024-09-11T00:31:01.251 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:01.246+0000 7f6872600640 1 -- 172.21.15.103:0/20348375 wait complete. 2024-09-11T00:31:02.500 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '56264023', total seconds elapsed: 31.7 2024-09-11T00:31:02.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:02 smithi103 ceph-mon[30578]: pgmap v638: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-11T00:31:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:02 smithi117 ceph-mon[35525]: pgmap v638: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-11T00:31:02.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:02 smithi183 ceph-mon[35934]: pgmap v638: 129 pgs: 129 active+clean; 115 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-11T00:31:04.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:04 smithi103 ceph-mon[30578]: pgmap v639: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-11T00:31:04.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:04 smithi117 ceph-mon[35525]: pgmap v639: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-11T00:31:04.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:04 smithi183 ceph-mon[35934]: pgmap v639: 129 pgs: 129 active+clean; 114 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 22 KiB/s wr, 15 op/s 2024-09-11T00:31:06.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:06 smithi103 ceph-mon[30578]: pgmap v640: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 225 KiB/s wr, 76 op/s 2024-09-11T00:31:06.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:06 smithi117 ceph-mon[35525]: pgmap v640: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 225 KiB/s wr, 76 op/s 2024-09-11T00:31:06.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:06 smithi183 ceph-mon[35934]: pgmap v640: 129 pgs: 129 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 225 KiB/s wr, 76 op/s 2024-09-11T00:31:08.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:08 smithi103 ceph-mon[30578]: pgmap v641: 129 pgs: 129 active+clean; 80 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 286 KiB/s wr, 92 op/s 2024-09-11T00:31:08.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:08 smithi117 ceph-mon[35525]: pgmap v641: 129 pgs: 129 active+clean; 80 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 286 KiB/s wr, 92 op/s 2024-09-11T00:31:08.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:08 smithi183 ceph-mon[35934]: pgmap v641: 129 pgs: 129 active+clean; 80 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 286 KiB/s wr, 92 op/s 2024-09-11T00:31:10.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:10 smithi103 ceph-mon[30578]: pgmap v642: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 541 KiB/s wr, 181 op/s 2024-09-11T00:31:10.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:10 smithi117 ceph-mon[35525]: pgmap v642: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 541 KiB/s wr, 181 op/s 2024-09-11T00:31:10.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:10 smithi183 ceph-mon[35934]: pgmap v642: 129 pgs: 129 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 541 KiB/s wr, 181 op/s 2024-09-11T00:31:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:12 smithi103 ceph-mon[30578]: pgmap v643: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 709 KiB/s wr, 240 op/s 2024-09-11T00:31:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:12 smithi117 ceph-mon[35525]: pgmap v643: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 709 KiB/s wr, 240 op/s 2024-09-11T00:31:12.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:12 smithi183 ceph-mon[35934]: pgmap v643: 129 pgs: 129 active+clean; 88 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 709 KiB/s wr, 240 op/s 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 -- 172.21.15.103:0/1568655482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa19410a910 msgr2=0x7fa19410cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 --2- 172.21.15.103:0/1568655482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa19410a910 0x7fa19410cd60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009ca0 tx=0x7fa18c02a8c0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 -- 172.21.15.103:0/1568655482 shutdown_connections 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 --2- 172.21.15.103:0/1568655482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa19410a910 0x7fa19410cd60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 --2- 172.21.15.103:0/1568655482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194075ac0 0x7fa194075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 --2- 172.21.15.103:0/1568655482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa194076f30 0x7fa194075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 -- 172.21.15.103:0/1568655482 >> 172.21.15.103:0/1568655482 conn(0x7fa1940fe250 msgr2=0x7fa194100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 -- 172.21.15.103:0/1568655482 shutdown_connections 2024-09-11T00:31:13.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.330+0000 7fa19c600640 1 -- 172.21.15.103:0/1568655482 wait complete. 2024-09-11T00:31:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.331+0000 7fa19c600640 1 Processor -- start 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.331+0000 7fa19c600640 1 -- start start 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19c600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194075ac0 0x7fa19419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19c600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa194076f30 0x7fa19419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19c600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19410a910 0x7fa19419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19c600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941107d0 con 0x7fa194076f30 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19c600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa194110650 con 0x7fa19410a910 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19c600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa194110950 con 0x7fa194075ac0 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19410a910 0x7fa19419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa199200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa194076f30 0x7fa19419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19410a910 0x7fa19419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44174/0 (socket says 172.21.15.103:44174) 2024-09-11T00:31:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 -- 172.21.15.103:0/716318789 learned_addr learned my addr 172.21.15.103:0/716318789 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa199c00640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194075ac0 0x7fa19419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 -- 172.21.15.103:0/716318789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194075ac0 msgr2=0x7fa19419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194075ac0 0x7fa19419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 -- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa194076f30 msgr2=0x7fa19419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa194076f30 0x7fa19419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa19a600640 1 -- 172.21.15.103:0/716318789 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19419ca10 con 0x7fa19410a910 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.332+0000 7fa199c00640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194075ac0 0x7fa19419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.333+0000 7fa19a600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19410a910 0x7fa19419c150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0097b0 tx=0x7fa18c031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.333+0000 7fa192000640 1 -- 172.21.15.103:0/716318789 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c003b70 con 0x7fa19410a910 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.333+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19419cca0 con 0x7fa19410a910 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.333+0000 7fa192000640 1 -- 172.21.15.103:0/716318789 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18c003d10 con 0x7fa19410a910 2024-09-11T00:31:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.333+0000 7fa192000640 1 -- 172.21.15.103:0/716318789 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c02fc10 con 0x7fa19410a910 2024-09-11T00:31:13.338 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.333+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1941a33d0 con 0x7fa19410a910 2024-09-11T00:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa192000640 1 -- 172.21.15.103:0/716318789 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa18c004040 con 0x7fa19410a910 2024-09-11T00:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 Processor -- start 2024-09-11T00:31:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 -- start start 2024-09-11T00:31:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fcbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fcc0c0 0x558015fce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558015f3c6b0 con 0x558015fcc0c0 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558015f2bae0 con 0x558015fc9790 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.337+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558015f2f8f0 con 0x558015fceb00 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fcbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fcc0c0 0x558015fce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fcbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44190/0 (socket says 172.21.15.103:44190) 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 -- 172.21.15.103:0/2189181767 learned_addr learned my addr 172.21.15.103:0/2189181767 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 -- 172.21.15.103:0/2189181767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 msgr2=0x558015fd0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.341 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 -- 172.21.15.103:0/2189181767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fcc0c0 msgr2=0x558015fce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fcc0c0 0x558015fce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 -- 172.21.15.103:0/2189181767 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558015ef8780 con 0x558015fc9790 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa199200640 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fcbb80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa180011850 tx=0x7fa1800097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bfb4079c89f8008 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa192000640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa17c0b5360 0x7fa17c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.338+0000 7fa186a00640 1 -- 172.21.15.103:0/2189181767 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180004050 con 0x558015fc9790 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.339+0000 7fa186a00640 1 -- 172.21.15.103:0/2189181767 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18000fc70 con 0x558015fc9790 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.339+0000 7fa186a00640 1 -- 172.21.15.103:0/2189181767 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180003850 con 0x558015fc9790 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.339+0000 7fa192000640 1 -- 172.21.15.103:0/716318789 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fa18c107400 con 0x7fa19410a910 2024-09-11T00:31:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.339+0000 7fa199c00640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa17c0b5360 0x7fa17c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.339+0000 7fa19f5ed740 1 -- 172.21.15.103:0/2189181767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 msgr2=0x558015fcbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.339+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fcbb80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa180011850 tx=0x7fa1800097f0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.339+0000 7fa199c00640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa17c0b5360 0x7fa17c0b7820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa188012640 tx=0x7fa188005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 -- 172.21.15.103:0/2189181767 shutdown_connections 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fcc0c0 0x558015fce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/2189181767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fcbb80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 -- 172.21.15.103:0/2189181767 >> 172.21.15.103:0/2189181767 conn(0x558015fc48e0 msgr2=0x558015fc5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 -- 172.21.15.103:0/2189181767 shutdown_connections 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 -- 172.21.15.103:0/2189181767 wait complete. 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 -- start start 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.340+0000 7fa19f5ed740 1 Processor -- start 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fc9790 0x558015fd2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fcc0c0 0x558015fd35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558015e63ab0 con 0x558015fc9790 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558015ed2570 con 0x558015fcc0c0 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558015fd1ae0 con 0x558015fceb00 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19f5ed740 10 client.0 _ll_register_callbacks cb 0x558015ee1100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa199c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fc9790 0x558015fd2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa199200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fcc0c0 0x558015fd35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa19a600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa199200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fcc0c0 0x558015fd35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44192/0 (socket says 172.21.15.103:44192) 2024-09-11T00:31:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.341+0000 7fa199200640 1 -- 172.21.15.103:0/184401452 learned_addr learned my addr 172.21.15.103:0/184401452 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19a600640 1 -- 172.21.15.103:0/184401452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fcc0c0 msgr2=0x558015fd35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19a600640 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fcc0c0 0x558015fd35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19a600640 1 -- 172.21.15.103:0/184401452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fc9790 msgr2=0x558015fd2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19a600640 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fc9790 0x558015fd2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19a600640 1 -- 172.21.15.103:0/184401452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558015c8d790 con 0x558015fceb00 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19a600640 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd4170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa18c104ee0 tx=0x7fa18c0c9790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a13f9f842b075abf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa17b400640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa17b400640 1 -- 172.21.15.103:0/184401452 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c0cd340 con 0x558015fceb00 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa18000fc70 con 0x558015fceb00 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa17b400640 1 -- 172.21.15.103:0/184401452 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fa18c107150 con 0x558015fceb00 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa17b400640 1 -- 172.21.15.103:0/184401452 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa18c0cdf30 con 0x558015fceb00 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19f5ed740 10 client.26269 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:13.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558015d06620 con 0x558015fceb00 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.342+0000 7fa17b400640 1 -- 172.21.15.103:0/184401452 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18000fc70 con 0x558015fceb00 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa17b400640 1 -- 172.21.15.103:0/184401452 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa18c0c9aa0 con 0x558015fceb00 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa19f5ed740 10 client.26269 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa19f5ed740 10 client.26269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa19f5ed740 20 client.26269 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558015fde790 0x558015fe0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa19f5ed740 4 client.26269 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558015f3e600 con 0x558015fde790 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.343+0000 7fa199c00640 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558015fde790 0x558015fe0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.344+0000 7fa199c00640 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558015fde790 0x558015fe0c10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbc88003217a3d72 server_cookie=5e6db6ac70327ad2 in_seq=0 out_seq=0 2024-09-11T00:31:13.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.344+0000 7fa17b400640 10 client.26269 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.344+0000 7fa17b400640 1 -- 172.21.15.103:0/184401452 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558015f3e600 con 0x558015fde790 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.344+0000 7fa17b400640 10 client.26269 handle_command_reply: tid=0 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.344+0000 7fa19f5ed740 1 client.26269 shutdown 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 msgr2=0x558015fd4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd4170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa18c104ee0 tx=0x7fa18c0c9790 comp rx=0 tx=0).stop 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 shutdown_connections 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558015fde790 0x558015fe0c10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fceb00 0x558015fd4170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa17b400640 0 client.26269 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fcc0c0 0x558015fd35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/184401452 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fc9790 0x558015fd2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 >> 172.21.15.103:0/184401452 conn(0x558015fc48e0 msgr2=0x558015fc3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 shutdown_connections 2024-09-11T00:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 1 -- 172.21.15.103:0/184401452 wait complete. 2024-09-11T00:31:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.345+0000 7fa19f5ed740 20 client.26269 trim_cache size 0 max 16384 2024-09-11T00:31:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 Processor -- start 2024-09-11T00:31:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 -- start start 2024-09-11T00:31:13.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fbfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fcf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941affd0 con 0x558015fceb00 2024-09-11T00:31:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941ad8a0 con 0x558015fc9790 2024-09-11T00:31:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.364+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941ab6f0 con 0x558015fcc0c0 2024-09-11T00:31:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fbfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fbfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44202/0 (socket says 172.21.15.103:44202) 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199c00640 1 -- 172.21.15.103:0/38783609 learned_addr learned my addr 172.21.15.103:0/38783609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199200640 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa19a600640 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fcf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199200640 1 -- 172.21.15.103:0/38783609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 msgr2=0x558015fbfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199200640 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fbfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199200640 1 -- 172.21.15.103:0/38783609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 msgr2=0x558015fcf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199200640 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fcf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199200640 1 -- 172.21.15.103:0/38783609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558015fcbef0 con 0x558015fcc0c0 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa199c00640 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fbfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.365+0000 7fa19a600640 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fcf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.366+0000 7fa199200640 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fceee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa180011850 tx=0x7fa180009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc03c14ae538fff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.366+0000 7fa186000640 1 -- 172.21.15.103:0/38783609 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180005040 con 0x558015fcc0c0 2024-09-11T00:31:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.366+0000 7fa186000640 1 -- 172.21.15.103:0/38783609 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18000f790 con 0x558015fcc0c0 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.366+0000 7fa186000640 1 -- 172.21.15.103:0/38783609 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1800048c0 con 0x558015fcc0c0 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.367+0000 7fa19f5ed740 1 -- 172.21.15.103:0/38783609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 msgr2=0x558015fceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.367+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fceee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa180011850 tx=0x7fa180009e50 comp rx=0 tx=0).stop 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.367+0000 7fa19f5ed740 1 -- 172.21.15.103:0/38783609 shutdown_connections 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.367+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fcf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.367+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fceee0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.367+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/38783609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fbfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.371 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.367+0000 7fa19f5ed740 1 -- 172.21.15.103:0/38783609 >> 172.21.15.103:0/38783609 conn(0x558015fc48e0 msgr2=0x558015fc5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.368+0000 7fa19f5ed740 1 -- 172.21.15.103:0/38783609 shutdown_connections 2024-09-11T00:31:13.374 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.368+0000 7fa19f5ed740 1 -- 172.21.15.103:0/38783609 wait complete. 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.368+0000 7fa19f5ed740 1 -- start start 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 1 Processor -- start 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fdaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fd97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fd9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941abd60 con 0x558015fceb00 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941ad1d0 con 0x558015fc9790 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1941ac430 con 0x558015fcc0c0 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa19f5ed740 10 client.0 _ll_register_callbacks cb 0x7fa18c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.369+0000 7fa199c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fdaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fdaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:44206/0 (socket says 172.21.15.103:44206) 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 -- 172.21.15.103:0/921052750 learned_addr learned my addr 172.21.15.103:0/921052750 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:13.375 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa19a600640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fd9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199200640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fd97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 -- 172.21.15.103:0/921052750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 msgr2=0x558015fd97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fd97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 -- 172.21.15.103:0/921052750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 msgr2=0x558015fd9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fd9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 -- 172.21.15.103:0/921052750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558015fd8310 con 0x558015fc9790 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199200640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fd97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa199c00640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fdaaf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa18800f940 tx=0x7fa1880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d58d78751f9f99b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.370+0000 7fa17b400640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.371+0000 7fa17b400640 1 -- 172.21.15.103:0/921052750 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188005020 con 0x558015fc9790 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.371+0000 7fa17b400640 1 -- 172.21.15.103:0/921052750 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7fa188018030 con 0x558015fc9790 2024-09-11T00:31:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.371+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa18000f790 con 0x558015fc9790 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.372+0000 7fa17b400640 1 -- 172.21.15.103:0/921052750 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa18803c050 con 0x558015fc9790 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.372+0000 7fa19f5ed740 10 client.26255 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.372+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55801604b930 con 0x558015fc9790 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa17b400640 1 -- 172.21.15.103:0/921052750 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa18000f790 con 0x558015fc9790 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa17b400640 1 -- 172.21.15.103:0/921052750 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa18801f5b0 con 0x558015fc9790 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa19f5ed740 10 client.26255 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa19f5ed740 10 client.26255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa19f5ed740 20 client.26255 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:13.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55801604c2c0 0x55801604e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa19f5ed740 4 client.26255 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.374+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558015f3a0b0 con 0x55801604c2c0 2024-09-11T00:31:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.375+0000 7fa199200640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55801604c2c0 0x55801604e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.375+0000 7fa199200640 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55801604c2c0 0x55801604e6e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b75c09f11b6872 server_cookie=cfd46145c12e55e8 in_seq=0 out_seq=0 2024-09-11T00:31:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.375+0000 7fa17b400640 10 client.26255 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:13.394 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa17b400640 1 -- 172.21.15.103:0/921052750 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558015f3a0b0 con 0x55801604c2c0 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa17b400640 10 client.26255 handle_command_reply: tid=0 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 client.26255 shutdown 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 msgr2=0x558015fdaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fdaaf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa18800f940 tx=0x7fa1880046c0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 shutdown_connections 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558015fceb00 0x558015fd9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55801604c2c0 0x55801604e6e0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558015fcc0c0 0x558015fd97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 --2- 172.21.15.103:0/921052750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558015fc9790 0x558015fdaaf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa17b400640 0 client.26255 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 >> 172.21.15.103:0/921052750 conn(0x558015fc48e0 msgr2=0x558015fc5140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 shutdown_connections 2024-09-11T00:31:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 1 -- 172.21.15.103:0/921052750 wait complete. 2024-09-11T00:31:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.392+0000 7fa19f5ed740 20 client.26255 trim_cache size 0 max 16384 2024-09-11T00:31:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:31:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:31:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:31:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa17c0b5360 msgr2=0x7fa17c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa17c0b5360 0x7fa17c0b7820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa188012640 tx=0x7fa188005f90 comp rx=0 tx=0).stop 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19410a910 msgr2=0x7fa19419c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19410a910 0x7fa19419c150 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa18c0097b0 tx=0x7fa18c031770 comp rx=0 tx=0).stop 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 shutdown_connections 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa17c0b5360 0x7fa17c0b7820 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19410a910 0x7fa19419c150 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa194076f30 0x7fa19419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 --2- 172.21.15.103:0/716318789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194075ac0 0x7fa19419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 >> 172.21.15.103:0/716318789 conn(0x7fa1940fe250 msgr2=0x7fa1941068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 shutdown_connections 2024-09-11T00:31:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:13.393+0000 7fa19c600640 1 -- 172.21.15.103:0/716318789 wait complete. 2024-09-11T00:31:14.664 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:14 smithi103 ceph-mon[30578]: pgmap v644: 129 pgs: 129 active+clean; 86 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 832 KiB/s wr, 299 op/s 2024-09-11T00:31:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:14 smithi117 ceph-mon[35525]: pgmap v644: 129 pgs: 129 active+clean; 86 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 832 KiB/s wr, 299 op/s 2024-09-11T00:31:14.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:14 smithi183 ceph-mon[35934]: pgmap v644: 129 pgs: 129 active+clean; 86 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 832 KiB/s wr, 299 op/s 2024-09-11T00:31:16.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:16 smithi103 ceph-mon[30578]: pgmap v645: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-11T00:31:16.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:16 smithi117 ceph-mon[35525]: pgmap v645: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-11T00:31:16.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:16 smithi183 ceph-mon[35934]: pgmap v645: 129 pgs: 129 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.1 MiB/s wr, 415 op/s 2024-09-11T00:31:18.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:18 smithi103 ceph-mon[30578]: pgmap v646: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 949 KiB/s wr, 377 op/s 2024-09-11T00:31:18.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:18 smithi117 ceph-mon[35525]: pgmap v646: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 949 KiB/s wr, 377 op/s 2024-09-11T00:31:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:18 smithi183 ceph-mon[35934]: pgmap v646: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 949 KiB/s wr, 377 op/s 2024-09-11T00:31:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:20 smithi103 ceph-mon[30578]: pgmap v647: 129 pgs: 129 active+clean; 81 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 456 op/s 2024-09-11T00:31:20.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:20 smithi117 ceph-mon[35525]: pgmap v647: 129 pgs: 129 active+clean; 81 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 456 op/s 2024-09-11T00:31:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:20 smithi183 ceph-mon[35934]: pgmap v647: 129 pgs: 129 active+clean; 81 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.1 MiB/s wr, 456 op/s 2024-09-11T00:31:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:22 smithi103 ceph-mon[30578]: pgmap v648: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-09-11T00:31:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:22 smithi117 ceph-mon[35525]: pgmap v648: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-09-11T00:31:22.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:22 smithi183 ceph-mon[35934]: pgmap v648: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 434 op/s 2024-09-11T00:31:23.595 INFO:tasks.workunit.client.0.smithi103.stdout:copying to largest ever: 0x63ef9ee 2024-09-11T00:31:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:24 smithi117 ceph-mon[35525]: pgmap v649: 129 pgs: 129 active+clean; 91 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 427 op/s 2024-09-11T00:31:24.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:24 smithi183 ceph-mon[35934]: pgmap v649: 129 pgs: 129 active+clean; 91 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 427 op/s 2024-09-11T00:31:24.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:24 smithi103 ceph-mon[30578]: pgmap v649: 129 pgs: 129 active+clean; 91 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.0 MiB/s wr, 427 op/s 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 -- 172.21.15.103:0/3344735608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c075ac0 msgr2=0x7f6a0c075f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 --2- 172.21.15.103:0/3344735608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c075f40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f69f8009900 tx=0x7f69f802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 -- 172.21.15.103:0/3344735608 shutdown_connections 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 --2- 172.21.15.103:0/3344735608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c10a910 0x7f6a0c10cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 --2- 172.21.15.103:0/3344735608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c075f40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 --2- 172.21.15.103:0/3344735608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a0c076f30 0x7f6a0c075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 -- 172.21.15.103:0/3344735608 >> 172.21.15.103:0/3344735608 conn(0x7f6a0c0fe250 msgr2=0x7f6a0c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 -- 172.21.15.103:0/3344735608 shutdown_connections 2024-09-11T00:31:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.524+0000 7f6a13e00640 1 -- 172.21.15.103:0/3344735608 wait complete. 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.525+0000 7f6a13e00640 1 Processor -- start 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.525+0000 7f6a13e00640 1 -- start start 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a0c076f30 0x7f6a0c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a13e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c10a910 0x7f6a0c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a13e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c110800 con 0x7f6a0c10a910 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a13e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c110680 con 0x7f6a0c075ac0 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a13e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c110980 con 0x7f6a0c076f30 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c19af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51556/0 (socket says 172.21.15.103:51556) 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a0c076f30 0x7f6a0c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 -- 172.21.15.103:0/968092289 learned_addr learned my addr 172.21.15.103:0/968092289 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c10a910 0x7f6a0c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 -- 172.21.15.103:0/968092289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a0c076f30 msgr2=0x7f6a0c19b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a0c076f30 0x7f6a0c19b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 -- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c10a910 msgr2=0x7f6a0c19c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c10a910 0x7f6a0c19c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 -- 172.21.15.103:0/968092289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c19cad0 con 0x7f6a0c075ac0 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.526+0000 7f6a11400640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c19af90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000e9f0 tx=0x7f6a0000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.527+0000 7f6a0a000640 1 -- 172.21.15.103:0/968092289 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0000cd80 con 0x7f6a0c075ac0 2024-09-11T00:31:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.527+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c19cdc0 con 0x7f6a0c075ac0 2024-09-11T00:31:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.527+0000 7f6a0a000640 1 -- 172.21.15.103:0/968092289 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a000054d0 con 0x7f6a0c075ac0 2024-09-11T00:31:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.527+0000 7f6a0a000640 1 -- 172.21.15.103:0/968092289 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a00010690 con 0x7f6a0c075ac0 2024-09-11T00:31:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.527+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1a3490 con 0x7f6a0c075ac0 2024-09-11T00:31:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 Processor -- start 2024-09-11T00:31:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 -- start start 2024-09-11T00:31:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f1533b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f15340c0 0x55c3f15364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1536b00 0x55c3f1538fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c3f14a46b0 con 0x55c3f1536b00 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c3f1493ae0 con 0x55c3f1531790 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a16e84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c3f14978f0 con 0x55c3f15340c0 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a11e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f1533b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a11e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f1533b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51568/0 (socket says 172.21.15.103:51568) 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a11e00640 1 -- 172.21.15.103:0/2212683542 learned_addr learned my addr 172.21.15.103:0/2212683542 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.530+0000 7f6a10a00640 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1536b00 0x55c3f1538fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f6a11e00640 1 -- 172.21.15.103:0/2212683542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f15340c0 msgr2=0x55c3f15364b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f6a11e00640 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f15340c0 0x55c3f15364b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f6a11e00640 1 -- 172.21.15.103:0/2212683542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1536b00 msgr2=0x55c3f1538fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:26.535 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f6a11e00640 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1536b00 0x55c3f1538fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f6a11e00640 1 -- 172.21.15.103:0/2212683542 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3f1460780 con 0x55c3f1531790 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f6a11e00640 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f1533b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009f90 tx=0x7f6a040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d3ee5f603dcb0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f6a0a000640 1 -- 172.21.15.103:0/968092289 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6a00004e90 con 0x7f6a0c075ac0 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f69fe600640 1 -- 172.21.15.103:0/2212683542 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04004b20 con 0x55c3f1531790 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f69fe600640 1 -- 172.21.15.103:0/2212683542 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a04004cc0 con 0x55c3f1531790 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.531+0000 7f69fe600640 1 -- 172.21.15.103:0/2212683542 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04005020 con 0x55c3f1531790 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 -- 172.21.15.103:0/2212683542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 msgr2=0x55c3f1533b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f1533b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009f90 tx=0x7f6a040118d0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 -- 172.21.15.103:0/2212683542 shutdown_connections 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1536b00 0x55c3f1538fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f15340c0 0x55c3f15364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2212683542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f1533b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.536 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 -- 172.21.15.103:0/2212683542 >> 172.21.15.103:0/2212683542 conn(0x55c3f152c8e0 msgr2=0x55c3f152d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 -- 172.21.15.103:0/2212683542 shutdown_connections 2024-09-11T00:31:26.537 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.532+0000 7f6a16e84740 1 -- 172.21.15.103:0/2212683542 wait complete. 2024-09-11T00:31:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 -- start start 2024-09-11T00:31:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 Processor -- start 2024-09-11T00:31:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1531790 0x55c3f153af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f15340c0 0x55c3f153b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f153c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55c3f13cbab0 con 0x55c3f1531790 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55c3f143a570 con 0x55c3f15340c0 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c3f1539ae0 con 0x55c3f1536b00 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a0a000640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69f40b5270 0x7f69f40b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.533+0000 7f6a16e84740 10 client.0 _ll_register_callbacks cb 0x55c3f1449100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f15340c0 0x55c3f153b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f15340c0 0x55c3f153b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:51584/0 (socket says 172.21.15.103:51584) 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 -- 172.21.15.103:0/2315575556 learned_addr learned my addr 172.21.15.103:0/2315575556 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11e00640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1531790 0x55c3f153af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a0a000640 1 -- 172.21.15.103:0/968092289 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6a000e63a0 con 0x7f6a0c075ac0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69f40b5270 0x7f69f40b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a10a00640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f153c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 -- 172.21.15.103:0/2315575556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 msgr2=0x55c3f153c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f153c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 -- 172.21.15.103:0/2315575556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1531790 msgr2=0x55c3f153af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1531790 0x55c3f153af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 -- 172.21.15.103:0/2315575556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f11f5790 con 0x55c3f15340c0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11e00640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1531790 0x55c3f153af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a10a00640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f153c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a11400640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f15340c0 0x55c3f153b5c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a000e3ab0 tx=0x7f6a000a8e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a595d4298bd27 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f69f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f69f2a00640 1 -- 172.21.15.103:0/2315575556 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000abee0 con 0x55c3f15340c0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f69f2a00640 1 -- 172.21.15.103:0/2315575556 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6a0001b530 con 0x55c3f15340c0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.534+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a04004cc0 con 0x55c3f15340c0 2024-09-11T00:31:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f6a11e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69f40b5270 0x7f69f40b7730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009f60 tx=0x7f6a04000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f69f2a00640 1 -- 172.21.15.103:0/2315575556 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6a0001b160 con 0x55c3f15340c0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f6a16e84740 10 client.26267 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c3f126e620 con 0x55c3f15340c0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f69f2a00640 1 -- 172.21.15.103:0/2315575556 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a04004cc0 con 0x55c3f15340c0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f69f2a00640 1 -- 172.21.15.103:0/2315575556 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6a000c3070 con 0x55c3f15340c0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f6a16e84740 10 client.26267 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.535+0000 7f6a16e84740 10 client.26267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.536+0000 7f6a16e84740 20 client.26267 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.536+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15467d0 0x55c3f1548bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.536+0000 7f6a16e84740 4 client.26267 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.536+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3f14a6600 con 0x55c3f15467d0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.536+0000 7f6a10a00640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15467d0 0x55c3f1548bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.536+0000 7f6a10a00640 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15467d0 0x55c3f1548bc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287b9572f135c65 server_cookie=a60b5fd05153d9a2 in_seq=0 out_seq=0 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.536+0000 7f69f2a00640 10 client.26267 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.537+0000 7f69f2a00640 1 -- 172.21.15.103:0/2315575556 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c3f14a6600 con 0x55c3f15467d0 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.537+0000 7f69f2a00640 10 client.26267 handle_command_reply: tid=0 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.537+0000 7f6a16e84740 1 client.26267 shutdown 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f15340c0 msgr2=0x55c3f153b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f15340c0 0x55c3f153b5c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6a000e3ab0 tx=0x7f6a000a8e80 comp rx=0 tx=0).stop 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 shutdown_connections 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15467d0 0x55c3f1548bc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f153c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f15340c0 0x55c3f153b5c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 --2- 172.21.15.103:0/2315575556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f1531790 0x55c3f153af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 >> 172.21.15.103:0/2315575556 conn(0x55c3f152c8e0 msgr2=0x55c3f152ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f69f2a00640 0 client.26267 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 shutdown_connections 2024-09-11T00:31:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 1 -- 172.21.15.103:0/2315575556 wait complete. 2024-09-11T00:31:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.538+0000 7f6a16e84740 20 client.26267 trim_cache size 0 max 16384 2024-09-11T00:31:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.563+0000 7f6a16e84740 1 Processor -- start 2024-09-11T00:31:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.563+0000 7f6a16e84740 1 -- start start 2024-09-11T00:31:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.564+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f15375f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.567 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.564+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f1527960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.564+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f1539ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.564+0000 7f6a16e84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1b0120 con 0x55c3f15340c0 2024-09-11T00:31:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.564+0000 7f6a16e84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ad9f0 con 0x55c3f1531790 2024-09-11T00:31:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.564+0000 7f6a16e84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ab840 con 0x55c3f1536b00 2024-09-11T00:31:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.565+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f1527960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.568 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.566+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f1527960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:33510/0 (socket says 172.21.15.103:33510) 2024-09-11T00:31:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.566+0000 7f6a10a00640 1 -- 172.21.15.103:0/3847232515 learned_addr learned my addr 172.21.15.103:0/3847232515 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.567+0000 7f6a10a00640 1 -- 172.21.15.103:0/3847232515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 msgr2=0x55c3f1539ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.566+0000 7f6a11e00640 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f1539ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.567+0000 7f6a10a00640 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f1539ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.567+0000 7f6a10a00640 1 -- 172.21.15.103:0/3847232515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 msgr2=0x55c3f15375f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:26.569 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.567+0000 7f6a10a00640 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f15375f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.567+0000 7f6a10a00640 1 -- 172.21.15.103:0/3847232515 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3f1536820 con 0x55c3f15340c0 2024-09-11T00:31:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.567+0000 7f6a11e00640 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f1539ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.568+0000 7f6a10a00640 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f1527960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a910 tx=0x7f69f80317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387822954c6d3cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:26.570 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.568+0000 7f69fdc00640 1 -- 172.21.15.103:0/3847232515 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f8035040 con 0x55c3f15340c0 2024-09-11T00:31:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.568+0000 7f69fdc00640 1 -- 172.21.15.103:0/3847232515 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f80035f0 con 0x55c3f15340c0 2024-09-11T00:31:26.571 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.568+0000 7f69fdc00640 1 -- 172.21.15.103:0/3847232515 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f803c680 con 0x55c3f15340c0 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 -- 172.21.15.103:0/3847232515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 msgr2=0x55c3f1527960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f1527960 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f69f802a910 tx=0x7f69f80317c0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 -- 172.21.15.103:0/3847232515 shutdown_connections 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1536b00 0x55c3f1539ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f1527960 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 --2- 172.21.15.103:0/3847232515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1531790 0x55c3f15375f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 -- 172.21.15.103:0/3847232515 >> 172.21.15.103:0/3847232515 conn(0x55c3f152c8e0 msgr2=0x55c3f1536ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 -- 172.21.15.103:0/3847232515 shutdown_connections 2024-09-11T00:31:26.572 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.570+0000 7f6a16e84740 1 -- 172.21.15.103:0/3847232515 wait complete. 2024-09-11T00:31:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 -- start start 2024-09-11T00:31:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 Processor -- start 2024-09-11T00:31:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1531790 0x55c3f1542330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f15428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1536b00 0x55c3f1540c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1abeb0 con 0x55c3f15340c0 2024-09-11T00:31:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ad320 con 0x55c3f1536b00 2024-09-11T00:31:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c1ac580 con 0x55c3f1531790 2024-09-11T00:31:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a16e84740 10 client.0 _ll_register_callbacks cb 0x7f6a0000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a11400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1531790 0x55c3f1542330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a10a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f15428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a11400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1531790 0x55c3f1542330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:43686/0 (socket says 172.21.15.103:43686) 2024-09-11T00:31:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.571+0000 7f6a11400640 1 -- 172.21.15.103:0/4222162988 learned_addr learned my addr 172.21.15.103:0/4222162988 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f6a11e00640 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1536b00 0x55c3f1540c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f6a11400640 1 -- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1536b00 msgr2=0x55c3f1540c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f6a11400640 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1536b00 0x55c3f1540c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f6a11400640 1 -- 172.21.15.103:0/4222162988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 msgr2=0x55c3f15428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f6a11400640 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f15428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f6a11400640 1 -- 172.21.15.103:0/4222162988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3f1548f30 con 0x55c3f1531790 2024-09-11T00:31:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f6a11400640 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1531790 0x55c3f1542330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000cfd0 tx=0x7f6a000e3dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb360454a1d84154 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f69f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:31:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f69f2a00640 1 -- 172.21.15.103:0/4222162988 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a000abee0 con 0x55c3f1531790 2024-09-11T00:31:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.572+0000 7f69f2a00640 1 -- 172.21.15.103:0/4222162988 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f6a000b5a70 con 0x55c3f1531790 2024-09-11T00:31:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69f80035f0 con 0x55c3f1531790 2024-09-11T00:31:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f69f2a00640 1 -- 172.21.15.103:0/4222162988 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6a000ac540 con 0x55c3f1531790 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 10 client.26284 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c3f15b3e50 con 0x55c3f1531790 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f69f2a00640 1 -- 172.21.15.103:0/4222162988 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f80035f0 con 0x55c3f1531790 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f69f2a00640 1 -- 172.21.15.103:0/4222162988 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6a000c3070 con 0x55c3f1531790 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 10 client.26284 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 10 client.26284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 20 client.26284 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15b4800 0x55c3f15b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 4 client.26284 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.573+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3f14a20b0 con 0x55c3f15b4800 2024-09-11T00:31:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.574+0000 7f6a10a00640 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15b4800 0x55c3f15b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.574+0000 7f69f2a00640 10 client.26284 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.574+0000 7f6a10a00640 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15b4800 0x55c3f15b6c20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5ecd39b46e8032 server_cookie=6209770caea97beb in_seq=0 out_seq=0 2024-09-11T00:31:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.589+0000 7f69f2a00640 1 -- 172.21.15.103:0/4222162988 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3f14a20b0 con 0x55c3f15b4800 2024-09-11T00:31:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.589+0000 7f69f2a00640 10 client.26284 handle_command_reply: tid=0 2024-09-11T00:31:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.590+0000 7f6a16e84740 1 client.26284 shutdown 2024-09-11T00:31:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.591+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1531790 msgr2=0x55c3f1542330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.591+0000 7f6a16e84740 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1531790 0x55c3f1542330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000cfd0 tx=0x7f6a000e3dd0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.591+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 shutdown_connections 2024-09-11T00:31:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.591+0000 7f6a16e84740 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55c3f1536b00 0x55c3f1540c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.591+0000 7f6a16e84740 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55c3f15340c0 0x55c3f15428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.591+0000 7f6a16e84740 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55c3f15b4800 0x55c3f15b6c20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.592+0000 7f6a16e84740 1 --2- 172.21.15.103:0/4222162988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3f1531790 0x55c3f1542330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.592+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 >> 172.21.15.103:0/4222162988 conn(0x55c3f152c8e0 msgr2=0x55c3f1536ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:26.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.592+0000 7f69f2a00640 0 client.26284 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.592+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 shutdown_connections 2024-09-11T00:31:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.593+0000 7f6a16e84740 1 -- 172.21.15.103:0/4222162988 wait complete. 2024-09-11T00:31:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.593+0000 7f6a16e84740 20 client.26284 trim_cache size 0 max 16384 2024-09-11T00:31:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:31:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:31:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:31:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:31:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.597+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69f40b5270 msgr2=0x7f69f40b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.597+0000 7f6a13e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69f40b5270 0x7f69f40b7730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009f60 tx=0x7f6a04000f40 comp rx=0 tx=0).stop 2024-09-11T00:31:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.597+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c075ac0 msgr2=0x7f6a0c19af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:26.599 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.597+0000 7f6a13e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c19af90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000e9f0 tx=0x7f6a0000edf0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.598+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 shutdown_connections 2024-09-11T00:31:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.598+0000 7f6a13e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f69f40b5270 0x7f69f40b7730 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.598+0000 7f6a13e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6a0c10a910 0x7f6a0c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.598+0000 7f6a13e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a0c076f30 0x7f6a0c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.598+0000 7f6a13e00640 1 --2- 172.21.15.103:0/968092289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a0c075ac0 0x7f6a0c19af90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.598+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 >> 172.21.15.103:0/968092289 conn(0x7f6a0c0fe250 msgr2=0x7f6a0c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:26.600 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.599+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 shutdown_connections 2024-09-11T00:31:26.601 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:26.599+0000 7f6a13e00640 1 -- 172.21.15.103:0/968092289 wait complete. 2024-09-11T00:31:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:26 smithi117 ceph-mon[35525]: pgmap v650: 129 pgs: 129 active+clean; 91 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 493 op/s 2024-09-11T00:31:26.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:26 smithi183 ceph-mon[35934]: pgmap v650: 129 pgs: 129 active+clean; 91 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 493 op/s 2024-09-11T00:31:26.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:26 smithi103 ceph-mon[30578]: pgmap v650: 129 pgs: 129 active+clean; 91 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 493 op/s 2024-09-11T00:31:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:28 smithi117 ceph-mon[35525]: pgmap v651: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 997 KiB/s wr, 390 op/s 2024-09-11T00:31:28.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:28 smithi183 ceph-mon[35934]: pgmap v651: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 997 KiB/s wr, 390 op/s 2024-09-11T00:31:28.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:28 smithi103 ceph-mon[30578]: pgmap v651: 129 pgs: 129 active+clean; 82 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 997 KiB/s wr, 390 op/s 2024-09-11T00:31:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:30 smithi117 ceph-mon[35525]: pgmap v652: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 442 op/s 2024-09-11T00:31:30.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:30 smithi183 ceph-mon[35934]: pgmap v652: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 442 op/s 2024-09-11T00:31:30.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:30 smithi103 ceph-mon[30578]: pgmap v652: 129 pgs: 129 active+clean; 74 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 442 op/s 2024-09-11T00:31:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:32 smithi117 ceph-mon[35525]: pgmap v653: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 435 op/s 2024-09-11T00:31:32.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:32 smithi183 ceph-mon[35934]: pgmap v653: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 435 op/s 2024-09-11T00:31:32.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:32 smithi103 ceph-mon[30578]: pgmap v653: 129 pgs: 129 active+clean; 79 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.1 MiB/s wr, 435 op/s 2024-09-11T00:31:34.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:34 smithi183 ceph-mon[35934]: pgmap v654: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-11T00:31:34.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:34 smithi103 ceph-mon[30578]: pgmap v654: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-11T00:31:34.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:34 smithi117 ceph-mon[35525]: pgmap v654: 129 pgs: 129 active+clean; 90 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 428 op/s 2024-09-11T00:31:36.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:36 smithi183 ceph-mon[35934]: pgmap v655: 129 pgs: 129 active+clean; 83 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 495 op/s 2024-09-11T00:31:36.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:36 smithi103 ceph-mon[30578]: pgmap v655: 129 pgs: 129 active+clean; 83 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 495 op/s 2024-09-11T00:31:36.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:36 smithi117 ceph-mon[35525]: pgmap v655: 129 pgs: 129 active+clean; 83 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 495 op/s 2024-09-11T00:31:38.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:38 smithi183 ceph-mon[35934]: pgmap v656: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 392 op/s 2024-09-11T00:31:38.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:38 smithi103 ceph-mon[30578]: pgmap v656: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 392 op/s 2024-09-11T00:31:38.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:38 smithi117 ceph-mon[35525]: pgmap v656: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.1 MiB/s wr, 392 op/s 2024-09-11T00:31:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 -- 172.21.15.103:0/3421182609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9084076f30 msgr2=0x7f9084075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 --2- 172.21.15.103:0/3421182609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9084076f30 0x7f9084075580 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9078009900 tx=0x7f907802a410 comp rx=0 tx=0).stop 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 -- 172.21.15.103:0/3421182609 shutdown_connections 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 --2- 172.21.15.103:0/3421182609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f908410a910 0x7f908410cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 --2- 172.21.15.103:0/3421182609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9084075ac0 0x7f9084075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 --2- 172.21.15.103:0/3421182609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9084076f30 0x7f9084075580 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 -- 172.21.15.103:0/3421182609 >> 172.21.15.103:0/3421182609 conn(0x7f90840fe250 msgr2=0x7f9084100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 -- 172.21.15.103:0/3421182609 shutdown_connections 2024-09-11T00:31:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.742+0000 7f908b600640 1 -- 172.21.15.103:0/3421182609 wait complete. 2024-09-11T00:31:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.743+0000 7f908b600640 1 Processor -- start 2024-09-11T00:31:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.743+0000 7f908b600640 1 -- start start 2024-09-11T00:31:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f908b600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9084075ac0 0x7f908419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f908b600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9084076f30 0x7f908419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f908b600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f908410a910 0x7f908419c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f908b600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9084110800 con 0x7f9084076f30 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f908b600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9084110680 con 0x7f908410a910 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f908b600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9084110980 con 0x7f9084075ac0 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9084076f30 0x7f908419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9084076f30 0x7f908419b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42726/0 (socket says 172.21.15.103:42726) 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 -- 172.21.15.103:0/1157639930 learned_addr learned my addr 172.21.15.103:0/1157639930 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9088c00640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9084075ac0 0x7f908419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9089600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f908410a910 0x7f908419c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 -- 172.21.15.103:0/1157639930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9084075ac0 msgr2=0x7f908419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9084075ac0 0x7f908419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 -- 172.21.15.103:0/1157639930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f908410a910 msgr2=0x7f908419c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f908410a910 0x7f908419c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.744+0000 7f9083e00640 1 -- 172.21.15.103:0/1157639930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908419cb30 con 0x7f9084076f30 2024-09-11T00:31:39.748 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.745+0000 7f9083e00640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9084076f30 0x7f908419b640 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f907000e9f0 tx=0x7f907000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.745+0000 7f9081600640 1 -- 172.21.15.103:0/1157639930 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907000cd80 con 0x7f9084076f30 2024-09-11T00:31:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.745+0000 7f9081600640 1 -- 172.21.15.103:0/1157639930 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90700054d0 con 0x7f9084076f30 2024-09-11T00:31:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.745+0000 7f9081600640 1 -- 172.21.15.103:0/1157639930 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070010690 con 0x7f9084076f30 2024-09-11T00:31:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.745+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908419ce20 con 0x7f9084076f30 2024-09-11T00:31:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.745+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90841a34f0 con 0x7f9084076f30 2024-09-11T00:31:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 Processor -- start 2024-09-11T00:31:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 -- start start 2024-09-11T00:31:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb790 0x5646b42bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.751 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5646b422e6b0 con 0x5646b42bb790 2024-09-11T00:31:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5646b421dae0 con 0x5646b42be0c0 2024-09-11T00:31:39.752 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.749+0000 7f908e3e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646b42218f0 con 0x5646b42c0b00 2024-09-11T00:31:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9081600640 1 -- 172.21.15.103:0/1157639930 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9070004e90 con 0x7f9084076f30 2024-09-11T00:31:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42c04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42698/0 (socket says 172.21.15.103:42698) 2024-09-11T00:31:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 -- 172.21.15.103:0/641027676 learned_addr learned my addr 172.21.15.103:0/641027676 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9089600640 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb790 0x5646b42bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9083e00640 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 -- 172.21.15.103:0/641027676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 msgr2=0x5646b42c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 -- 172.21.15.103:0/641027676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb790 msgr2=0x5646b42bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb790 0x5646b42bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9088c00640 1 -- 172.21.15.103:0/641027676 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b41ea780 con 0x5646b42be0c0 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.750+0000 7f9089600640 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb790 0x5646b42bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.751+0000 7f9088c00640 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42c04b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9078009900 tx=0x7f9078005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f454eaef14c802d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.751+0000 7f9075c00640 1 -- 172.21.15.103:0/641027676 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078035030 con 0x5646b42be0c0 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.751+0000 7f9075c00640 1 -- 172.21.15.103:0/641027676 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9078040070 con 0x5646b42be0c0 2024-09-11T00:31:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.751+0000 7f9075c00640 1 -- 172.21.15.103:0/641027676 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078003810 con 0x5646b42be0c0 2024-09-11T00:31:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 -- 172.21.15.103:0/641027676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 msgr2=0x5646b42c04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.757 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42c04b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9078009900 tx=0x7f9078005de0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 -- 172.21.15.103:0/641027676 shutdown_connections 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42c04b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 --2- 172.21.15.103:0/641027676 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb790 0x5646b42bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 -- 172.21.15.103:0/641027676 >> 172.21.15.103:0/641027676 conn(0x5646b42b68e0 msgr2=0x5646b42b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 -- 172.21.15.103:0/641027676 shutdown_connections 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 -- 172.21.15.103:0/641027676 wait complete. 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f9081600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f906c0b52f0 0x7f906c0b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f9081600640 1 -- 172.21.15.103:0/1157639930 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f90700e63a0 con 0x7f9084076f30 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f908e3e4740 1 -- start start 2024-09-11T00:31:39.758 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.752+0000 7f9088c00640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f906c0b52f0 0x7f906c0b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 1 Processor -- start 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb790 0x5646b42c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5646b4155ab0 con 0x5646b42be0c0 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5646b41c4570 con 0x5646b42bb790 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646b42c3a80 con 0x5646b42c0b00 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f908e3e4740 10 client.0 _ll_register_callbacks cb 0x5646b41d3100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f9088c00640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f906c0b52f0 0x7f906c0b77b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f90780096c0 tx=0x7f907803e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f9088c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb790 0x5646b42c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f9088c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb790 0x5646b42c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42706/0 (socket says 172.21.15.103:42706) 2024-09-11T00:31:39.759 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.753+0000 7f9088c00640 1 -- 172.21.15.103:0/1797538694 learned_addr learned my addr 172.21.15.103:0/1797538694 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9083e00640 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9089600640 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9088c00640 1 -- 172.21.15.103:0/1797538694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 msgr2=0x5646b42c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9088c00640 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9088c00640 1 -- 172.21.15.103:0/1797538694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 msgr2=0x5646b42c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9088c00640 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9088c00640 1 -- 172.21.15.103:0/1797538694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b3f7f790 con 0x5646b42bb790 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f9088c00640 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb790 0x5646b42c4e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90780042a0 tx=0x7f907802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e1877eeb87465cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f906a000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f906a000640 1 -- 172.21.15.103:0/1797538694 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078060060 con 0x5646b42bb790 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9078040070 con 0x5646b42bb790 2024-09-11T00:31:39.760 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.754+0000 7f906a000640 1 -- 172.21.15.103:0/1797538694 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f90780035f0 con 0x5646b42bb790 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.755+0000 7f906a000640 1 -- 172.21.15.103:0/1797538694 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9078035050 con 0x5646b42bb790 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.755+0000 7f908e3e4740 10 client.26282 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.755+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5646b3ff8620 con 0x5646b42bb790 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.755+0000 7f906a000640 1 -- 172.21.15.103:0/1797538694 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9078040070 con 0x5646b42bb790 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.756+0000 7f906a000640 1 -- 172.21.15.103:0/1797538694 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f907803c710 con 0x5646b42bb790 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.757+0000 7f908e3e4740 10 client.26282 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.757+0000 7f908e3e4740 10 client.26282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.757+0000 7f908e3e4740 20 client.26282 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.757+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b42d02b0 0x5646b42d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.757+0000 7f908e3e4740 4 client.26282 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.757+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646b4230600 con 0x5646b42d02b0 2024-09-11T00:31:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.757+0000 7f9083e00640 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b42d02b0 0x5646b42d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.760+0000 7f9083e00640 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b42d02b0 0x5646b42d26a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68b027dfe754da61 server_cookie=7044a0ab9210f62 in_seq=0 out_seq=0 2024-09-11T00:31:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.760+0000 7f906a000640 10 client.26282 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.761+0000 7f906a000640 1 -- 172.21.15.103:0/1797538694 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646b4230600 con 0x5646b42d02b0 2024-09-11T00:31:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.761+0000 7f906a000640 10 client.26282 handle_command_reply: tid=0 2024-09-11T00:31:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.762+0000 7f908e3e4740 1 client.26282 shutdown 2024-09-11T00:31:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.768+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb790 msgr2=0x5646b42c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.768+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb790 0x5646b42c4e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90780042a0 tx=0x7f907802a410 comp rx=0 tx=0).stop 2024-09-11T00:31:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 shutdown_connections 2024-09-11T00:31:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b42d02b0 0x5646b42d26a0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1797538694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb790 0x5646b42c4e10 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f90780042a0 tx=0x7f907802a410 comp rx=0 tx=0).stop 2024-09-11T00:31:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 >> 172.21.15.103:0/1797538694 conn(0x5646b42b68e0 msgr2=0x5646b42b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f906a000640 0 client.26282 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 shutdown_connections 2024-09-11T00:31:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.769+0000 7f908e3e4740 1 -- 172.21.15.103:0/1797538694 wait complete. 2024-09-11T00:31:39.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.770+0000 7f908e3e4740 20 client.26282 trim_cache size 0 max 16384 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 Processor -- start 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 -- start start 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb300 0x5646b42b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90841b0140 con 0x5646b42be0c0 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90841ada10 con 0x5646b42bb300 2024-09-11T00:31:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f908e3e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90841ab860 con 0x5646b42c0b00 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.795+0000 7f9089600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb300 0x5646b42b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb300 0x5646b42b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42712/0 (socket says 172.21.15.103:42712) 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 -- 172.21.15.103:0/1082927285 learned_addr learned my addr 172.21.15.103:0/1082927285 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9088c00640 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9083e00640 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 -- 172.21.15.103:0/1082927285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 msgr2=0x5646b42b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 -- 172.21.15.103:0/1082927285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 msgr2=0x5646b42b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 -- 172.21.15.103:0/1082927285 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b42c4930 con 0x5646b42bb300 2024-09-11T00:31:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9088c00640 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.796+0000 7f9089600640 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb300 0x5646b42b8710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f907c003b30 tx=0x7f907c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8772a4ccd17a9f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.797+0000 7f9075200640 1 -- 172.21.15.103:0/1082927285 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c020070 con 0x5646b42bb300 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.797+0000 7f9075200640 1 -- 172.21.15.103:0/1082927285 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f907c0108e0 con 0x5646b42bb300 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.797+0000 7f9075200640 1 -- 172.21.15.103:0/1082927285 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c01b9b0 con 0x5646b42bb300 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.797+0000 7f908e3e4740 1 -- 172.21.15.103:0/1082927285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb300 msgr2=0x5646b42b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.797+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb300 0x5646b42b8710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f907c003b30 tx=0x7f907c004040 comp rx=0 tx=0).stop 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 -- 172.21.15.103:0/1082927285 shutdown_connections 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42bb300 0x5646b42b8710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1082927285 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42be0c0 0x5646b42b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 -- 172.21.15.103:0/1082927285 >> 172.21.15.103:0/1082927285 conn(0x5646b42b68e0 msgr2=0x5646b42babf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 -- 172.21.15.103:0/1082927285 shutdown_connections 2024-09-11T00:31:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 -- 172.21.15.103:0/1082927285 wait complete. 2024-09-11T00:31:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.798+0000 7f908e3e4740 1 -- start start 2024-09-11T00:31:39.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 1 Processor -- start 2024-09-11T00:31:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb300 0x5646b42cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42cad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90841abed0 con 0x5646b42bb300 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f90841ad340 con 0x5646b42be0c0 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90841ac5a0 con 0x5646b42c0b00 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.799+0000 7f908e3e4740 10 client.0 _ll_register_callbacks cb 0x7f907c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9088c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb300 0x5646b42cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9083e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9083e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:42716/0 (socket says 172.21.15.103:42716) 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9083e00640 1 -- 172.21.15.103:0/1050054053 learned_addr learned my addr 172.21.15.103:0/1050054053 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9089600640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9083e00640 1 -- 172.21.15.103:0/1050054053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 msgr2=0x5646b42cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9083e00640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.800+0000 7f9083e00640 1 -- 172.21.15.103:0/1050054053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb300 msgr2=0x5646b42cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f9083e00640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb300 0x5646b42cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f9083e00640 1 -- 172.21.15.103:0/1050054053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b42cb2d0 con 0x5646b42be0c0 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f9089600640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f9088c00640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb300 0x5646b42cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f9083e00640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42cc6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f907000c6a0 tx=0x7f90700ac600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97d12dda6cf077ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f906a000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f906a000640 1 -- 172.21.15.103:0/1050054053 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070005680 con 0x5646b42be0c0 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.801+0000 7f906a000640 1 -- 172.21.15.103:0/1050054053 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f90700051b0 con 0x5646b42be0c0 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f907c0108e0 con 0x5646b42be0c0 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f906a000640 1 -- 172.21.15.103:0/1050054053 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f90700ace70 con 0x5646b42be0c0 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 10 client.26288 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5646b433d920 con 0x5646b42be0c0 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f906a000640 1 -- 172.21.15.103:0/1050054053 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f907c0108e0 con 0x5646b42be0c0 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f906a000640 1 -- 172.21.15.103:0/1050054053 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f90700c3080 con 0x5646b42be0c0 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 10 client.26288 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 20 client.26288 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b433e300 0x5646b4340720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 4 client.26288 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.802+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646b422c0b0 con 0x5646b433e300 2024-09-11T00:31:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.803+0000 7f9088c00640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b433e300 0x5646b4340720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.803+0000 7f9088c00640 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b433e300 0x5646b4340720 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26be5dcf9d08f785 server_cookie=e78c706fcc254e60 in_seq=0 out_seq=0 2024-09-11T00:31:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.803+0000 7f906a000640 10 client.26288 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f906a000640 1 -- 172.21.15.103:0/1050054053 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646b422c0b0 con 0x5646b433e300 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f906a000640 10 client.26288 handle_command_reply: tid=0 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 client.26288 shutdown 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 msgr2=0x5646b42cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42cc6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f907000c6a0 tx=0x7f90700ac600 comp rx=0 tx=0).stop 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 shutdown_connections 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5646b433e300 0x5646b4340720 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646b42c0b00 0x5646b42cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5646b42be0c0 0x5646b42cc6a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f906a000640 0 client.26288 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 --2- 172.21.15.103:0/1050054053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646b42bb300 0x5646b42cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 >> 172.21.15.103:0/1050054053 conn(0x5646b42b68e0 msgr2=0x5646b42b7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 shutdown_connections 2024-09-11T00:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.818+0000 7f908e3e4740 1 -- 172.21.15.103:0/1050054053 wait complete. 2024-09-11T00:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908e3e4740 20 client.26288 trim_cache size 0 max 16384 2024-09-11T00:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:31:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f906c0b52f0 msgr2=0x7f906c0b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f906c0b52f0 0x7f906c0b77b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f90780096c0 tx=0x7f907803e040 comp rx=0 tx=0).stop 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9084076f30 msgr2=0x7f908419b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9084076f30 0x7f908419b640 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f907000e9f0 tx=0x7f907000edf0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 shutdown_connections 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f906c0b52f0 0x7f906c0b77b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f908410a910 0x7f908419c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9084076f30 0x7f908419b640 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 --2- 172.21.15.103:0/1157639930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9084075ac0 0x7f908419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 >> 172.21.15.103:0/1157639930 conn(0x7f90840fe250 msgr2=0x7f90840fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.819+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 shutdown_connections 2024-09-11T00:31:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:39.820+0000 7f908b600640 1 -- 172.21.15.103:0/1157639930 wait complete. 2024-09-11T00:31:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:40 smithi183 ceph-mon[35934]: pgmap v657: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 439 op/s 2024-09-11T00:31:40.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:40 smithi103 ceph-mon[30578]: pgmap v657: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 439 op/s 2024-09-11T00:31:40.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:40 smithi117 ceph-mon[35525]: pgmap v657: 129 pgs: 129 active+clean; 84 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.2 MiB/s wr, 439 op/s 2024-09-11T00:31:42.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:42 smithi183 ceph-mon[35934]: pgmap v658: 129 pgs: 129 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-11T00:31:42.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:42 smithi103 ceph-mon[30578]: pgmap v658: 129 pgs: 129 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-11T00:31:42.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:42 smithi117 ceph-mon[35525]: pgmap v658: 129 pgs: 129 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-11T00:31:44.721 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:44 smithi103 ceph-mon[30578]: pgmap v659: 129 pgs: 129 active+clean; 80 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 427 op/s 2024-09-11T00:31:44.739 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:44 smithi117 ceph-mon[35525]: pgmap v659: 129 pgs: 129 active+clean; 80 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 427 op/s 2024-09-11T00:31:44.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:44 smithi183 ceph-mon[35934]: pgmap v659: 129 pgs: 129 active+clean; 80 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 427 op/s 2024-09-11T00:31:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:46 smithi183 ceph-mon[35934]: pgmap v660: 129 pgs: 129 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 491 op/s 2024-09-11T00:31:46.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:46 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:31:46.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:46 smithi103 ceph-mon[30578]: pgmap v660: 129 pgs: 129 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 491 op/s 2024-09-11T00:31:46.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:46 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:31:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:46 smithi117 ceph-mon[35525]: pgmap v660: 129 pgs: 129 active+clean; 82 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 491 op/s 2024-09-11T00:31:46.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:46 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:31:47.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:31:47.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:31:47.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:31:47.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:31:47.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:31:47.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:31:47.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:31:47.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:31:47.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:31:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:48 smithi183 ceph-mon[35934]: pgmap v661: 129 pgs: 129 active+clean; 83 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 391 op/s 2024-09-11T00:31:48.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:48 smithi103 ceph-mon[30578]: pgmap v661: 129 pgs: 129 active+clean; 83 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 391 op/s 2024-09-11T00:31:48.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:48 smithi117 ceph-mon[35525]: pgmap v661: 129 pgs: 129 active+clean; 83 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 391 op/s 2024-09-11T00:31:50.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:50 smithi183 ceph-mon[35934]: pgmap v662: 129 pgs: 129 active+clean; 79 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-11T00:31:50.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:50 smithi103 ceph-mon[30578]: pgmap v662: 129 pgs: 129 active+clean; 79 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-11T00:31:50.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:50 smithi117 ceph-mon[35525]: pgmap v662: 129 pgs: 129 active+clean; 79 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-11T00:31:52.273 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.4 2024-09-11T00:31:52.274 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:31:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:52 smithi103 ceph-mon[30578]: pgmap v663: 129 pgs: 129 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-11T00:31:52.762 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:31:52.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:52 smithi183 ceph-mon[35934]: pgmap v663: 129 pgs: 129 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-11T00:31:52.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:52 smithi117 ceph-mon[35525]: pgmap v663: 129 pgs: 129 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.2 MiB/s wr, 456 op/s 2024-09-11T00:31:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 -- 172.21.15.103:0/3623258954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc102970 msgr2=0x7f9ccc102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/3623258954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc102970 0x7f9ccc102d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009900 tx=0x7f9cbc02a400 comp rx=0 tx=0).stop 2024-09-11T00:31:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 -- 172.21.15.103:0/3623258954 shutdown_connections 2024-09-11T00:31:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/3623258954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ccc104480 0x7f9ccc10ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/3623258954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ccc103ac0 0x7f9ccc103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/3623258954 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc102970 0x7f9ccc102d70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 -- 172.21.15.103:0/3623258954 >> 172.21.15.103:0/3623258954 conn(0x7f9ccc0fe250 msgr2=0x7f9ccc100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 -- 172.21.15.103:0/3623258954 shutdown_connections 2024-09-11T00:31:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.968+0000 7f9cd1c00640 1 -- 172.21.15.103:0/3623258954 wait complete. 2024-09-11T00:31:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.969+0000 7f9cd1c00640 1 Processor -- start 2024-09-11T00:31:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cd1c00640 1 -- start start 2024-09-11T00:31:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cd1c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ccc102970 0x7f9ccc076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cd1c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc103ac0 0x7f9ccc075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cd1c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ccc104480 0x7f9ccc075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cd1c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10e610 con 0x7f9ccc104480 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cd1c00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10e490 con 0x7f9ccc103ac0 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cd1c00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc10e790 con 0x7f9ccc102970 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cca000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc103ac0 0x7f9ccc075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9ccaa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ccc102970 0x7f9ccc076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cca000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc103ac0 0x7f9ccc075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41858/0 (socket says 172.21.15.103:41858) 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9cca000640 1 -- 172.21.15.103:0/571113000 learned_addr learned my addr 172.21.15.103:0/571113000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.970+0000 7f9ccb400640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ccc104480 0x7f9ccc075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cca000640 1 -- 172.21.15.103:0/571113000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ccc102970 msgr2=0x7f9ccc076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cca000640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ccc102970 0x7f9ccc076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cca000640 1 -- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ccc104480 msgr2=0x7f9ccc075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cca000640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ccc104480 0x7f9ccc075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cca000640 1 -- 172.21.15.103:0/571113000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ccc0762f0 con 0x7f9ccc103ac0 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9ccb400640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ccc104480 0x7f9ccc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ccc102970 0x7f9ccc076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cca000640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc103ac0 0x7f9ccc075540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cc3400640 1 -- 172.21.15.103:0/571113000 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800cd80 con 0x7f9ccc103ac0 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cc3400640 1 -- 172.21.15.103:0/571113000 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cb80108e0 con 0x7f9ccc103ac0 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.971+0000 7f9cc3400640 1 -- 172.21.15.103:0/571113000 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8018980 con 0x7f9ccc103ac0 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.972+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ccc1a2e90 con 0x7f9ccc103ac0 2024-09-11T00:31:52.975 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.972+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ccc1a3300 con 0x7f9ccc103ac0 2024-09-11T00:31:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 Processor -- start 2024-09-11T00:31:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 -- start start 2024-09-11T00:31:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207042b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 0x5652070454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x565207047fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565206fb36b0 con 0x5652070430c0 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565206fa2ae0 con 0x565207040790 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565206fa68f0 con 0x565207045b00 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.976+0000 7f9ccb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207042b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccaa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 0x5652070454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207042b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41872/0 (socket says 172.21.15.103:41872) 2024-09-11T00:31:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 -- 172.21.15.103:0/250280618 learned_addr learned my addr 172.21.15.103:0/250280618 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 -- 172.21.15.103:0/250280618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 msgr2=0x565207047fc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 --2- 172.21.15.103:0/250280618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x565207047fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 -- 172.21.15.103:0/250280618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 msgr2=0x5652070454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 --2- 172.21.15.103:0/250280618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 0x5652070454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 -- 172.21.15.103:0/250280618 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565206f6f780 con 0x565207040790 2024-09-11T00:31:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9cc3400640 1 -- 172.21.15.103:0/571113000 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f9cb8010430 con 0x7f9ccc103ac0 2024-09-11T00:31:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9ccb400640 1 --2- 172.21.15.103:0/250280618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207042b80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009fd0 tx=0x7f9cc4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e30c6eab00ce7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9cb7e00640 1 -- 172.21.15.103:0/250280618 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4011700 con 0x565207040790 2024-09-11T00:31:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9cb7e00640 1 -- 172.21.15.103:0/250280618 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc4011d50 con 0x565207040790 2024-09-11T00:31:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.977+0000 7f9cb7e00640 1 -- 172.21.15.103:0/250280618 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400e3c0 con 0x565207040790 2024-09-11T00:31:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.978+0000 7f9cd4c87740 1 -- 172.21.15.103:0/250280618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 msgr2=0x565207042b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.978+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/250280618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207042b80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009fd0 tx=0x7f9cc4010860 comp rx=0 tx=0).stop 2024-09-11T00:31:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 -- 172.21.15.103:0/250280618 shutdown_connections 2024-09-11T00:31:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/250280618 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x565207047fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/250280618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 0x5652070454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/250280618 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207042b80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 -- 172.21.15.103:0/250280618 >> 172.21.15.103:0/250280618 conn(0x56520703b8e0 msgr2=0x56520703c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 -- 172.21.15.103:0/250280618 shutdown_connections 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 -- 172.21.15.103:0/250280618 wait complete. 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cc3400640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cb00b5270 0x7f9cb00b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 -- start start 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cc3400640 1 -- 172.21.15.103:0/571113000 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f9cb80e62e0 con 0x7f9ccc103ac0 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9ccb400640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cb00b5270 0x7f9cb00b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.979+0000 7f9cd4c87740 1 Processor -- start 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207040790 0x565207049ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x56520704a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x56520704b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565206edaab0 con 0x565207040790 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x565206f49570 con 0x5652070430c0 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565207048ae0 con 0x565207045b00 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cd4c87740 10 client.0 _ll_register_callbacks cb 0x565206f58100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9ccb400640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cb00b5270 0x7f9cb00b7730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4005bf0 tx=0x7f9cc4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cca000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x56520704a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cca000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x56520704a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41882/0 (socket says 172.21.15.103:41882) 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9cca000640 1 -- 172.21.15.103:0/911794774 learned_addr learned my addr 172.21.15.103:0/911794774 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9ccb400640 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x56520704b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.980+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207040790 0x565207049ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cca000640 1 -- 172.21.15.103:0/911794774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 msgr2=0x56520704b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cca000640 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x56520704b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cca000640 1 -- 172.21.15.103:0/911794774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207040790 msgr2=0x565207049ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cca000640 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207040790 0x565207049ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cca000640 1 -- 172.21.15.103:0/911794774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565206d04790 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cca000640 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x56520704a670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80097b0 tx=0x7f9cb80ac060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f806c83b2a90740 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cb4c00640 1 -- 172.21.15.103:0/911794774 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80abb70 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cc4011d50 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cb4c00640 1 -- 172.21.15.103:0/911794774 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f9cb80d34d0 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.981+0000 7f9cb4c00640 1 -- 172.21.15.103:0/911794774 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9cb80ace10 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 10 client.26300 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x565206d7d620 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cb4c00640 1 -- 172.21.15.103:0/911794774 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cc4011d50 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cb4c00640 1 -- 172.21.15.103:0/911794774 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9cb80a8060 con 0x5652070430c0 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 10 client.26300 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 20 client.26300 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565207055730 0x565207057b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 4 client.26300 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565206fb5600 con 0x565207055730 2024-09-11T00:31:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.982+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565207055730 0x565207057b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.983+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565207055730 0x565207057b70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6bd711923be560 server_cookie=f7649e180001132 in_seq=0 out_seq=0 2024-09-11T00:31:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.983+0000 7f9cb4c00640 10 client.26300 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cb4c00640 1 -- 172.21.15.103:0/911794774 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565206fb5600 con 0x565207055730 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cb4c00640 10 client.26300 handle_command_reply: tid=0 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 client.26300 shutdown 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 msgr2=0x56520704a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x56520704a670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80097b0 tx=0x7f9cb80ac060 comp rx=0 tx=0).stop 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 shutdown_connections 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x565207055730 0x565207057b70 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x56520704b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x56520704a670 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/911794774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207040790 0x565207049ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cb4c00640 0 client.26300 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 >> 172.21.15.103:0/911794774 conn(0x56520703b8e0 msgr2=0x56520703aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 shutdown_connections 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 1 -- 172.21.15.103:0/911794774 wait complete. 2024-09-11T00:31:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:52.984+0000 7f9cd4c87740 20 client.26300 trim_cache size 0 max 16384 2024-09-11T00:31:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 Processor -- start 2024-09-11T00:31:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 -- start start 2024-09-11T00:31:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207040790 0x565207048ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x565207049420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207045b00 0x56520703d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1b01c0 con 0x565207045b00 2024-09-11T00:31:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1ada90 con 0x5652070430c0 2024-09-11T00:31:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.000+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1ab8e0 con 0x565207040790 2024-09-11T00:31:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.001+0000 7f9ccb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207045b00 0x56520703d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.001+0000 7f9ccb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207045b00 0x56520703d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45784/0 (socket says 172.21.15.103:45784) 2024-09-11T00:31:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.001+0000 7f9ccb400640 1 -- 172.21.15.103:0/870896958 learned_addr learned my addr 172.21.15.103:0/870896958 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.001+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207040790 0x565207048ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9cca000640 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x565207049420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9ccaa00640 1 -- 172.21.15.103:0/870896958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 msgr2=0x565207049420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x565207049420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9ccaa00640 1 -- 172.21.15.103:0/870896958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207045b00 msgr2=0x56520703d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207045b00 0x56520703d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9ccaa00640 1 -- 172.21.15.103:0/870896958 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565207037270 con 0x565207040790 2024-09-11T00:31:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9cca000640 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x565207049420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:31:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207040790 0x565207048ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02a920 tx=0x7f9cbc005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=718e9cd3cd7b8e02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:53.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.002+0000 7f9cb7400640 1 -- 172.21.15.103:0/870896958 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc04a070 con 0x565207040790 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.003+0000 7f9cb7400640 1 -- 172.21.15.103:0/870896958 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cbc003bc0 con 0x565207040790 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.003+0000 7f9cb7400640 1 -- 172.21.15.103:0/870896958 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc035040 con 0x565207040790 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 -- 172.21.15.103:0/870896958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207040790 msgr2=0x565207048ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207040790 0x565207048ee0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc02a920 tx=0x7f9cbc005a20 comp rx=0 tx=0).stop 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 -- 172.21.15.103:0/870896958 shutdown_connections 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x565207045b00 0x56520703d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5652070430c0 0x565207049420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/870896958 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207040790 0x565207048ee0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 -- 172.21.15.103:0/870896958 >> 172.21.15.103:0/870896958 conn(0x56520703b8e0 msgr2=0x565207045f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.004+0000 7f9cd4c87740 1 -- 172.21.15.103:0/870896958 shutdown_connections 2024-09-11T00:31:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.005+0000 7f9cd4c87740 1 -- 172.21.15.103:0/870896958 wait complete. 2024-09-11T00:31:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.005+0000 7f9cd4c87740 1 -- start start 2024-09-11T00:31:53.007 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.005+0000 7f9cd4c87740 1 Processor -- start 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207050b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 0x5652070510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9cd4c87740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x565207051630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1abf50 con 0x5652070430c0 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1ad3c0 con 0x565207040790 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9cd4c87740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9ccc1ac620 con 0x565207045b00 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9cd4c87740 10 client.0 _ll_register_callbacks cb 0x7f9cb8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9ccaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207050b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9ccaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207050b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41896/0 (socket says 172.21.15.103:41896) 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.006+0000 7f9ccaa00640 1 -- 172.21.15.103:0/3753407184 learned_addr learned my addr 172.21.15.103:0/3753407184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.007+0000 7f9ccb400640 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x565207051630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.007+0000 7f9ccaa00640 1 -- 172.21.15.103:0/3753407184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 msgr2=0x565207051630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.007+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x565207051630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.007+0000 7f9ccaa00640 1 -- 172.21.15.103:0/3753407184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 msgr2=0x5652070510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.007+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 0x5652070510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.007+0000 7f9ccaa00640 1 -- 172.21.15.103:0/3753407184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56520704f9e0 con 0x565207040790 2024-09-11T00:31:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.008+0000 7f9ccaa00640 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207050b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005ab0 tx=0x7f9cbc005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64dd77a77c40fca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.008+0000 7f9cb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:31:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.008+0000 7f9cb4c00640 1 -- 172.21.15.103:0/3753407184 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc067070 con 0x565207040790 2024-09-11T00:31:53.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.008+0000 7f9cb4c00640 1 -- 172.21.15.103:0/3753407184 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9150+0+0 (secure 0 0 0) 0x7f9cbc031530 con 0x565207040790 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cbc003bc0 con 0x565207040790 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cb4c00640 1 -- 172.21.15.103:0/3753407184 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9cbc073050 con 0x565207040790 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 10 client.26303 fetch_fsmap learned FSMap version 43 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5652070c2b80 con 0x565207040790 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cb4c00640 1 -- 172.21.15.103:0/3753407184 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9cbc003bc0 con 0x565207040790 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cb4c00640 1 -- 172.21.15.103:0/3753407184 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9cbc03c440 con 0x565207040790 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 10 client.26303 fetch_fsmap finished waiting for FSMap version 43 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 20 client.26303 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5652070c3500 0x5652070c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 4 client.26303 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:31:53.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.009+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565206fb10b0 con 0x5652070c3500 2024-09-11T00:31:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.010+0000 7f9cca000640 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5652070c3500 0x5652070c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.010+0000 7f9cca000640 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5652070c3500 0x5652070c5920 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdb5ef319000dc4e server_cookie=31255ff8d4eadeb0 in_seq=0 out_seq=0 2024-09-11T00:31:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.010+0000 7f9cb4c00640 10 client.26303 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cb4c00640 1 -- 172.21.15.103:0/3753407184 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565206fb10b0 con 0x5652070c3500 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cb4c00640 10 client.26303 handle_command_reply: tid=0 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 client.26303 shutdown 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 msgr2=0x565207050b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207050b90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005ab0 tx=0x7f9cbc005e80 comp rx=0 tx=0).stop 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 shutdown_connections 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5652070c3500 0x5652070c5920 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565207045b00 0x565207051630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5652070430c0 0x5652070510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cb4c00640 0 client.26303 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 --2- 172.21.15.103:0/3753407184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x565207040790 0x565207050b90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 >> 172.21.15.103:0/3753407184 conn(0x56520703b8e0 msgr2=0x565207049ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 shutdown_connections 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.021+0000 7f9cd4c87740 1 -- 172.21.15.103:0/3753407184 wait complete. 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.022+0000 7f9cd4c87740 20 client.26303 trim_cache size 0 max 16384 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:31:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:31:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.022+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cb00b5270 msgr2=0x7f9cb00b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.022+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cb00b5270 0x7f9cb00b7730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4005bf0 tx=0x7f9cc4005fd0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.022+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc103ac0 msgr2=0x7f9ccc075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.022+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc103ac0 0x7f9ccc075540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800e9f0 tx=0x7f9cb800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 shutdown_connections 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f9cb00b5270 0x7f9cb00b7730 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ccc104480 0x7f9ccc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ccc103ac0 0x7f9ccc075540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 --2- 172.21.15.103:0/571113000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9ccc102970 0x7f9ccc076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 >> 172.21.15.103:0/571113000 conn(0x7f9ccc0fe250 msgr2=0x7f9ccc104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 shutdown_connections 2024-09-11T00:31:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:31:53.023+0000 7f9cd1c00640 1 -- 172.21.15.103:0/571113000 wait complete. 2024-09-11T00:31:53.479 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setuser ceph since I am not root 2024-09-11T00:31:53.480 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setgroup ceph since I am not root 2024-09-11T00:31:53.480 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-mds[65455]: -- 172.21.15.183:0/2498173582 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559be4385860 con 0x559be4362800 2024-09-11T00:31:53.480 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: starting mds.f at 2024-09-11T00:31:53.617 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:53 smithi103 ceph-mon[30578]: daemon mds.f restarted 2024-09-11T00:31:53.617 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:53 smithi103 ceph-mon[30578]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T00:31:53.617 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:53 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6834/4278110475,v1:172.21.15.183:6835/4278110475] up:boot 2024-09-11T00:31:53.617 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:53 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:31:53.617 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:53 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 -- 172.21.15.103:0/1027874433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50102160 msgr2=0x7f7e5010eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 --2- 172.21.15.103:0/1027874433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50102160 0x7f7e5010eeb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009900 tx=0x7f7e3802a480 comp rx=0 tx=0).stop 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 -- 172.21.15.103:0/1027874433 shutdown_connections 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 --2- 172.21.15.103:0/1027874433 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e5010f3f0 0x7f7e501117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 --2- 172.21.15.103:0/1027874433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e50102160 0x7f7e5010eeb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 --2- 172.21.15.103:0/1027874433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e50101840 0x7f7e50101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 -- 172.21.15.103:0/1027874433 >> 172.21.15.103:0/1027874433 conn(0x7f7e500fd730 msgr2=0x7f7e500ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 -- 172.21.15.103:0/1027874433 shutdown_connections 2024-09-11T00:31:53.644 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.639+0000 7f7e56000640 1 -- 172.21.15.103:0/1027874433 wait complete. 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.641+0000 7f7e56000640 1 Processor -- start 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.641+0000 7f7e56000640 1 -- start start 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.642+0000 7f7e56000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e50101840 0x7f7e5010ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.642+0000 7f7e56000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e50102160 0x7f7e5010be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.642+0000 7f7e56000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010f3f0 0x7f7e5010c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.642+0000 7f7e56000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50114f80 con 0x7f7e50101840 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.642+0000 7f7e56000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50114e00 con 0x7f7e50102160 2024-09-11T00:31:53.645 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.642+0000 7f7e56000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e50115100 con 0x7f7e5010f3f0 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.642+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010f3f0 0x7f7e5010c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.643+0000 7f7e4fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010f3f0 0x7f7e5010c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44934/0 (socket says 172.21.15.103:44934) 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.643+0000 7f7e4fe00640 1 -- 172.21.15.103:0/3841681317 learned_addr learned my addr 172.21.15.103:0/3841681317 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.643+0000 7f7e4fe00640 1 -- 172.21.15.103:0/3841681317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e50102160 msgr2=0x7f7e5010be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.643+0000 7f7e4fe00640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e50102160 0x7f7e5010be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.643+0000 7f7e4fe00640 1 -- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e50101840 msgr2=0x7f7e5010ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.643+0000 7f7e4fe00640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e50101840 0x7f7e5010ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.646 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.643+0000 7f7e4fe00640 1 -- 172.21.15.103:0/3841681317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5010cc40 con 0x7f7e5010f3f0 2024-09-11T00:31:53.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.644+0000 7f7e4fe00640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010f3f0 0x7f7e5010c3a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eb10 tx=0x7f7e4800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:53.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.644+0000 7f7e4cc00640 1 -- 172.21.15.103:0/3841681317 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48004b20 con 0x7f7e5010f3f0 2024-09-11T00:31:53.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.644+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e501a1560 con 0x7f7e5010f3f0 2024-09-11T00:31:53.647 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.644+0000 7f7e4cc00640 1 -- 172.21.15.103:0/3841681317 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e48004cc0 con 0x7f7e5010f3f0 2024-09-11T00:31:53.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.645+0000 7f7e4cc00640 1 -- 172.21.15.103:0/3841681317 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e48010670 con 0x7f7e5010f3f0 2024-09-11T00:31:53.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.645+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e501a19f0 con 0x7f7e5010f3f0 2024-09-11T00:31:53.649 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.647+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c0052c0 con 0x7f7e5010f3f0 2024-09-11T00:31:53.653 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.650+0000 7f7e4cc00640 1 -- 172.21.15.103:0/3841681317 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f7e480108f0 con 0x7f7e5010f3f0 2024-09-11T00:31:53.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.653+0000 7f7e4cc00640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7e340b53a0 0x7f7e340b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:53.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.653+0000 7f7e4cc00640 1 -- 172.21.15.103:0/3841681317 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f7e480e6340 con 0x7f7e5010f3f0 2024-09-11T00:31:53.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.654+0000 7f7e4f400640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7e340b53a0 0x7f7e340b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:53.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.654+0000 7f7e4cc00640 1 -- 172.21.15.103:0/3841681317 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7e480ace70 con 0x7f7e5010f3f0 2024-09-11T00:31:53.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.655+0000 7f7e4f400640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7e340b53a0 0x7f7e340b7860 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7e400058c0 tx=0x7f7e40005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-mon[35934]: daemon mds.f restarted 2024-09-11T00:31:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-mon[35934]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T00:31:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6834/4278110475,v1:172.21.15.183:6835/4278110475] up:boot 2024-09-11T00:31:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:31:53.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:53 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:31:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:53 smithi117 ceph-mon[35525]: daemon mds.f restarted 2024-09-11T00:31:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:53 smithi117 ceph-mon[35525]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T00:31:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:53 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6834/4278110475,v1:172.21.15.183:6835/4278110475] up:boot 2024-09-11T00:31:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:53 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:31:53.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:53 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:31:53.942 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.940+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e2c0055b0 con 0x7f7e5010f3f0 2024-09-11T00:31:53.943 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.941+0000 7f7e4cc00640 1 -- 172.21.15.103:0/3841681317 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11933 (secure 0 0 0) 0x7f7e4807a1c0 con 0x7f7e5010f3f0 2024-09-11T00:31:53.944 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:31:53.944 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":44,"btime":"2024-09-11T00:31:53:238018+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:31:53.238002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26323":{"gid":26323,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4278110475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4278110475},{"type":"v1","addr":"172.21.15.183:6835","nonce":4278110475}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:31:53.944 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 44 2024-09-11T00:31:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7e340b53a0 msgr2=0x7f7e340b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7e340b53a0 0x7f7e340b7860 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7e400058c0 tx=0x7f7e40005850 comp rx=0 tx=0).stop 2024-09-11T00:31:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010f3f0 msgr2=0x7f7e5010c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:53.946 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010f3f0 0x7f7e5010c3a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e4800eb10 tx=0x7f7e4800c2d0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 shutdown_connections 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f7e340b53a0 0x7f7e340b7860 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e5010f3f0 0x7f7e5010c3a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e50102160 0x7f7e5010be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 --2- 172.21.15.103:0/3841681317 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7e50101840 0x7f7e5010ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.944+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 >> 172.21.15.103:0/3841681317 conn(0x7f7e500fd730 msgr2=0x7f7e501115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.945+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 shutdown_connections 2024-09-11T00:31:53.947 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:53.945+0000 7f7e56000640 1 -- 172.21.15.103:0/3841681317 wait complete. 2024-09-11T00:31:54.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:54 smithi103 ceph-mon[30578]: pgmap v664: 129 pgs: 129 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-09-11T00:31:54.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:54 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3841681317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:54.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:54 smithi183 ceph-mon[35934]: pgmap v664: 129 pgs: 129 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-09-11T00:31:54.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:54 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3841681317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:54.905 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0' 2024-09-11T00:31:54.905 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0 2024-09-11T00:31:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:54 smithi117 ceph-mon[35525]: pgmap v664: 129 pgs: 129 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.0 MiB/s wr, 422 op/s 2024-09-11T00:31:54.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:54 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3841681317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:31:55.450 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:31:56.462 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.460+0000 7fa3ba800640 1 -- 172.21.15.103:0/2126089881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 msgr2=0x7fa3b410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.460+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2126089881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b410cb30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009900 tx=0x7fa39c02a410 comp rx=0 tx=0).stop 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.461+0000 7fa3ba800640 1 -- 172.21.15.103:0/2126089881 shutdown_connections 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.461+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2126089881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b410d100 0x7fa3b410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.461+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2126089881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b410cb30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.461+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2126089881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b4103b50 0x7fa3b4103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.461+0000 7fa3ba800640 1 -- 172.21.15.103:0/2126089881 >> 172.21.15.103:0/2126089881 conn(0x7fa3b40fd700 msgr2=0x7fa3b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.461+0000 7fa3ba800640 1 -- 172.21.15.103:0/2126089881 shutdown_connections 2024-09-11T00:31:56.463 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.461+0000 7fa3ba800640 1 -- 172.21.15.103:0/2126089881 wait complete. 2024-09-11T00:31:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.462+0000 7fa3ba800640 1 Processor -- start 2024-09-11T00:31:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.462+0000 7fa3ba800640 1 -- start start 2024-09-11T00:31:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3ba800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b4103b50 0x7fa3b419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.464 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3ba800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3ba800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b410d100 0x7fa3b41a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3ba800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4112db0 con 0x7fa3b4103b50 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3ba800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4112c30 con 0x7fa3b4104550 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3ba800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b4112f30 con 0x7fa3b410d100 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41932/0 (socket says 172.21.15.103:41932) 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3abe00640 1 -- 172.21.15.103:0/2752620687 learned_addr learned my addr 172.21.15.103:0/2752620687 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3b3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b410d100 0x7fa3b41a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.465 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.463+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b4103b50 0x7fa3b419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3abe00640 1 -- 172.21.15.103:0/2752620687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b410d100 msgr2=0x7fa3b41a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b410d100 0x7fa3b41a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3abe00640 1 -- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b4103b50 msgr2=0x7fa3b419b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b4103b50 0x7fa3b419b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3abe00640 1 -- 172.21.15.103:0/2752620687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b41a1590 con 0x7fa3b4104550 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b4103b50 0x7fa3b419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b410d100 0x7fa3b41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:56.466 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.464+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b419ba20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005ab0 tx=0x7fa39c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.465+0000 7fa3b0c00640 1 -- 172.21.15.103:0/2752620687 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c03f070 con 0x7fa3b4104550 2024-09-11T00:31:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.465+0000 7fa3b0c00640 1 -- 172.21.15.103:0/2752620687 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c031b60 con 0x7fa3b4104550 2024-09-11T00:31:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.465+0000 7fa3b0c00640 1 -- 172.21.15.103:0/2752620687 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c035030 con 0x7fa3b4104550 2024-09-11T00:31:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.465+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b41a1820 con 0x7fa3b4104550 2024-09-11T00:31:56.467 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.465+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b41a1c20 con 0x7fa3b4104550 2024-09-11T00:31:56.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 Processor -- start 2024-09-11T00:31:56.471 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 -- start start 2024-09-11T00:31:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67e0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ec4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67eca30 0x5574e67eee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574e6755ca0 con 0x5574e67e0aa0 2024-09-11T00:31:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574e67450d0 con 0x5574e67eca30 2024-09-11T00:31:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.469+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574e6748ee0 con 0x5574e67e0180 2024-09-11T00:31:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67e0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44960/0 (socket says 172.21.15.103:44960) 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 -- 172.21.15.103:0/4092283184 learned_addr learned my addr 172.21.15.103:0/4092283184 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 -- 172.21.15.103:0/4092283184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67eca30 msgr2=0x5574e67eee20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/4092283184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67eca30 0x5574e67eee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 -- 172.21.15.103:0/4092283184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 msgr2=0x5574e67ec4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/4092283184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ec4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.470+0000 7fa3b3e00640 1 -- 172.21.15.103:0/4092283184 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574e66f7400 con 0x5574e67e0180 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.471+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/4092283184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67e0560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0117e0 tx=0x7fa3ac011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=567575b1192ef772 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.473+0000 7fa3a8c00640 1 -- 172.21.15.103:0/4092283184 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac011310 con 0x5574e67e0180 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.473+0000 7fa3a8c00640 1 -- 172.21.15.103:0/4092283184 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ac003e20 con 0x5574e67e0180 2024-09-11T00:31:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.473+0000 7fa3a8c00640 1 -- 172.21.15.103:0/4092283184 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0044b0 con 0x5574e67e0180 2024-09-11T00:31:56.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.474+0000 7fa3bd22a740 1 -- 172.21.15.103:0/4092283184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 msgr2=0x5574e67e0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:56.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.474+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/4092283184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67e0560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0117e0 tx=0x7fa3ac011ca0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.475+0000 7fa3b0c00640 1 -- 172.21.15.103:0/2752620687 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fa39c03b460 con 0x7fa3b4104550 2024-09-11T00:31:56.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.477+0000 7fa3b0c00640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3980b5360 0x7fa3980b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.479 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.478+0000 7fa3b0c00640 1 -- 172.21.15.103:0/2752620687 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa39c1068e0 con 0x7fa3b4104550 2024-09-11T00:31:56.480 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.479+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3980b5360 0x7fa3980b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.480+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3980b5360 0x7fa3980b7820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0049d0 tx=0x7fa3ac0114c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:56.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.481+0000 7fa3bd22a740 1 -- 172.21.15.103:0/4092283184 shutdown_connections 2024-09-11T00:31:56.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.481+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/4092283184 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67eca30 0x5574e67eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.481+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/4092283184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ec4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.482 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.481+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/4092283184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67e0560 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.481+0000 7fa3bd22a740 1 -- 172.21.15.103:0/4092283184 >> 172.21.15.103:0/4092283184 conn(0x5574e67ddfb0 msgr2=0x5574e67df150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:56.483 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.482+0000 7fa3bd22a740 1 -- 172.21.15.103:0/4092283184 shutdown_connections 2024-09-11T00:31:56.485 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.482+0000 7fa3bd22a740 1 -- 172.21.15.103:0/4092283184 wait complete. 2024-09-11T00:31:56.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.483+0000 7fa3bd22a740 1 -- start start 2024-09-11T00:31:56.486 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.485+0000 7fa3bd22a740 1 Processor -- start 2024-09-11T00:31:56.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.486+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67e0180 0x5574e67ec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.486+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.486+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67eca30 0x5574e67e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.486+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5574e65a67a0 con 0x5574e67eca30 2024-09-11T00:31:56.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.486+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5574e66eaa10 con 0x5574e67e0180 2024-09-11T00:31:56.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.486+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574e67ef6b0 con 0x5574e67e0aa0 2024-09-11T00:31:56.488 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.487+0000 7fa3bd22a740 10 client.0 _ll_register_callbacks cb 0x5574e66fabe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:56.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.487+0000 7fa3b3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67e0180 0x5574e67ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.489 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.487+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.487+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44972/0 (socket says 172.21.15.103:44972) 2024-09-11T00:31:56.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.488+0000 7fa3abe00640 1 -- 172.21.15.103:0/2680974004 learned_addr learned my addr 172.21.15.103:0/2680974004 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:56.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.487+0000 7fa3b3400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67e0180 0x5574e67ec360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41940/0 (socket says 172.21.15.103:41940) 2024-09-11T00:31:56.490 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.488+0000 7fa3b3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67eca30 0x5574e67e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.490+0000 7fa3abe00640 1 -- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67e0180 msgr2=0x5574e67ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:56.491 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.490+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67e0180 0x5574e67ec360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.490+0000 7fa3abe00640 1 -- 172.21.15.103:0/2680974004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67eca30 msgr2=0x5574e67e9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:56.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.490+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67eca30 0x5574e67e9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.490+0000 7fa3abe00640 1 -- 172.21.15.103:0/2680974004 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574e64b2b90 con 0x5574e67e0aa0 2024-09-11T00:31:56.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.490+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67e0180 0x5574e67ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:56.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.490+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67eca30 0x5574e67e9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:56.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.491+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67e97b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0c9f80 tx=0x7fa39c0cc660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bedfa4579dc15a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:56.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.491+0000 7fa3a1600640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:31:56.493 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.492+0000 7fa3a1600640 1 -- 172.21.15.103:0/2680974004 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0cd410 con 0x5574e67e0aa0 2024-09-11T00:31:56.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.492+0000 7fa3a1600640 1 -- 172.21.15.103:0/2680974004 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa39c0c8f80 con 0x5574e67e0aa0 2024-09-11T00:31:56.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.493+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3ac003e20 con 0x5574e67e0aa0 2024-09-11T00:31:56.494 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.494+0000 7fa3a1600640 1 -- 172.21.15.103:0/2680974004 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa39c0ccd10 con 0x5574e67e0aa0 2024-09-11T00:31:56.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.494+0000 7fa3bd22a740 10 client.26341 fetch_fsmap learned FSMap version 44 2024-09-11T00:31:56.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.494+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5574e67fb080 con 0x5574e67e0aa0 2024-09-11T00:31:56.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3a1600640 1 -- 172.21.15.103:0/2680974004 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3ac003e20 con 0x5574e67e0aa0 2024-09-11T00:31:56.496 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3a1600640 1 -- 172.21.15.103:0/2680974004 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fa39c0d5e10 con 0x5574e67e0aa0 2024-09-11T00:31:56.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3bd22a740 10 client.26341 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:31:56.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3bd22a740 10 client.26341 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:31:56.497 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3bd22a740 20 client.26341 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:56.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67fc580 0x5574e67fe970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3bd22a740 4 client.26341 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:31:56.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.495+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574e6757bf0 con 0x5574e67fc580 2024-09-11T00:31:56.498 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.496+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67fc580 0x5574e67fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.499 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.498+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67fc580 0x5574e67fe970 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e65571f2ce05c5ff server_cookie=8602e0ae35d0514d in_seq=0 out_seq=0 2024-09-11T00:31:56.500 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.498+0000 7fa3a1600640 10 client.26341 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:56.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.499+0000 7fa3a1600640 1 -- 172.21.15.103:0/2680974004 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5574e6757bf0 con 0x5574e67fc580 2024-09-11T00:31:56.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.499+0000 7fa3a1600640 10 client.26341 handle_command_reply: tid=0 2024-09-11T00:31:56.501 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.500+0000 7fa3bd22a740 1 client.26341 shutdown 2024-09-11T00:31:56.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 msgr2=0x5574e67e97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:56.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67e97b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0c9f80 tx=0x7fa39c0cc660 comp rx=0 tx=0).stop 2024-09-11T00:31:56.502 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 shutdown_connections 2024-09-11T00:31:56.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67eca30 0x5574e67e9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67fc580 0x5574e67fe970 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67e97b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2680974004 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67e0180 0x5574e67ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.501+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 >> 172.21.15.103:0/2680974004 conn(0x5574e67ddfb0 msgr2=0x5574e67e3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:56.503 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.502+0000 7fa3a1600640 0 client.26341 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:56.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.502+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 shutdown_connections 2024-09-11T00:31:56.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.502+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2680974004 wait complete. 2024-09-11T00:31:56.504 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.502+0000 7fa3bd22a740 20 client.26341 trim_cache size 0 max 16384 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 Processor -- start 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 -- start start 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67dd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0180 0x5574e67ede20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67ee360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41b8970 con 0x5574e67e0180 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41b6240 con 0x5574e67ddfb0 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.525+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41b40f0 con 0x5574e67e0aa0 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0180 0x5574e67ede20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0180 0x5574e67ede20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:45850/0 (socket says 172.21.15.103:45850) 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 -- 172.21.15.103:0/1621024742 learned_addr learned my addr 172.21.15.103:0/1621024742 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:56.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3b3400640 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67dd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67ee360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 -- 172.21.15.103:0/1621024742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 msgr2=0x5574e67ee360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:56.528 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67ee360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 -- 172.21.15.103:0/1621024742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 msgr2=0x5574e67dd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67dd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.526+0000 7fa3abe00640 1 -- 172.21.15.103:0/1621024742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574e67e08d0 con 0x5574e67e0180 2024-09-11T00:31:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.527+0000 7fa3abe00640 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0180 0x5574e67ede20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0cca10 tx=0x7fa39c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a1e48beec163f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.527+0000 7fa3a3e00640 1 -- 172.21.15.103:0/1621024742 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0cd410 con 0x5574e67e0180 2024-09-11T00:31:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.527+0000 7fa3a3e00640 1 -- 172.21.15.103:0/1621024742 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c0d17f0 con 0x5574e67e0180 2024-09-11T00:31:56.529 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.527+0000 7fa3a3e00640 1 -- 172.21.15.103:0/1621024742 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0ccf70 con 0x5574e67e0180 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 -- 172.21.15.103:0/1621024742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0180 msgr2=0x5574e67ede20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0180 0x5574e67ede20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0cca10 tx=0x7fa39c02a410 comp rx=0 tx=0).stop 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 -- 172.21.15.103:0/1621024742 shutdown_connections 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0aa0 0x5574e67ee360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0180 0x5574e67ede20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/1621024742 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67dd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 -- 172.21.15.103:0/1621024742 >> 172.21.15.103:0/1621024742 conn(0x5574e67e5120 msgr2=0x5574e67f7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 -- 172.21.15.103:0/1621024742 shutdown_connections 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.528+0000 7fa3bd22a740 1 -- 172.21.15.103:0/1621024742 wait complete. 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 -- start start 2024-09-11T00:31:56.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 Processor -- start 2024-09-11T00:31:56.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41b4740 con 0x5574e67e0aa0 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41b5b70 con 0x5574e67ddfb0 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b41b4dd0 con 0x5574e67e0180 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3bd22a740 10 client.0 _ll_register_callbacks cb 0x7fa39c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.529+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3abe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:44984/0 (socket says 172.21.15.103:44984) 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3abe00640 1 -- 172.21.15.103:0/2573146360 learned_addr learned my addr 172.21.15.103:0/2573146360 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.533 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3400640 1 -- 172.21.15.103:0/2573146360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 msgr2=0x5574e67f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3400640 1 -- 172.21.15.103:0/2573146360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 msgr2=0x5574e67ef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3400640 1 -- 172.21.15.103:0/2573146360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574e67efef0 con 0x5574e67ddfb0 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3e00640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3b3400640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67f49b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5574e67ff170 tx=0x7fa3a4005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3c01731c18a2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3a1600640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.530+0000 7fa3a1600640 1 -- 172.21.15.103:0/2573146360 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4012650 con 0x5574e67ddfb0 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa39c0d17f0 con 0x5574e67ddfb0 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3a1600640 1 -- 172.21.15.103:0/2573146360 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fa3a40127f0 con 0x5574e67ddfb0 2024-09-11T00:31:56.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3a1600640 1 -- 172.21.15.103:0/2573146360 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa3a402c050 con 0x5574e67ddfb0 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 10 client.26327 fetch_fsmap learned FSMap version 44 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5574e6869d10 con 0x5574e67ddfb0 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3a1600640 1 -- 172.21.15.103:0/2573146360 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa39c0d17f0 con 0x5574e67ddfb0 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3a1600640 1 -- 172.21.15.103:0/2573146360 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fa3a4024860 con 0x5574e67ddfb0 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 10 client.26327 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 10 client.26327 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 20 client.26327 populate_metadata read hostname 'smithi103' 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67eca30 0x5574e686c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 4 client.26327 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}] 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.531+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}) -- 0x5574e67536a0 con 0x5574e67eca30 2024-09-11T00:31:56.535 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.532+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67eca30 0x5574e686c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:31:56.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.532+0000 7fa3abe00640 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67eca30 0x5574e686c570 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=827bb5e685d9d984 server_cookie=5b8cae0cf3bc5b43 in_seq=0 out_seq=0 2024-09-11T00:31:56.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:56.532+0000 7fa3a1600640 10 client.26327 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:56.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:56 smithi183 ceph-mon[35934]: pgmap v666: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.0 MiB/s wr, 421 op/s 2024-09-11T00:31:56.913 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:56 smithi103 ceph-mon[30578]: pgmap v666: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.0 MiB/s wr, 421 op/s 2024-09-11T00:31:56.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:56 smithi117 ceph-mon[35525]: pgmap v666: 129 pgs: 129 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.0 MiB/s wr, 421 op/s 2024-09-11T00:31:57.518 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.516+0000 7fa3a1600640 1 -- 172.21.15.103:0/2573146360 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5574e67536a0 con 0x5574e67eca30 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.516+0000 7fa3a1600640 10 client.26327 handle_command_reply: tid=0 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.516+0000 7fa3bd22a740 1 client.26327 shutdown 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 msgr2=0x5574e67f49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67f49b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x5574e67ff170 tx=0x7fa3a4005410 comp rx=0 tx=0).stop 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 shutdown_connections 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5574e67e0aa0 0x5574e67ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5574e67eca30 0x5574e686c570 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574e67e0180 0x5574e67f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 --2- 172.21.15.103:0/2573146360 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5574e67ddfb0 0x5574e67f49b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.519 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.517+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 >> 172.21.15.103:0/2573146360 conn(0x5574e67e5120 msgr2=0x5574e67dd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:57.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.518+0000 7fa3a1600640 0 client.26327 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:31:57.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.518+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 shutdown_connections 2024-09-11T00:31:57.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.518+0000 7fa3bd22a740 1 -- 172.21.15.103:0/2573146360 wait complete. 2024-09-11T00:31:57.520 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.519+0000 7fa3bd22a740 20 client.26327 trim_cache size 0 max 16384 2024-09-11T00:31:57.521 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:31:57.521 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 44, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 26, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "19d99768": { 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 26, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 148.4, 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:31:57.522 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "QUIESCED", 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:31:57.523 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.520+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3980b5360 msgr2=0x7fa3980b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.520+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3980b5360 0x7fa3980b7820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0049d0 tx=0x7fa3ac0114c0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 msgr2=0x7fa3b419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b419ba20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa39c005ab0 tx=0x7fa39c003c10 comp rx=0 tx=0).stop 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 shutdown_connections 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fa3980b5360 0x7fa3980b7820 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3b410d100 0x7fa3b41a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b4104550 0x7fa3b419ba20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 --2- 172.21.15.103:0/2752620687 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b4103b50 0x7fa3b419b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:31:57.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 >> 172.21.15.103:0/2752620687 conn(0x7fa3b40fd700 msgr2=0x7fa3b410f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:31:57.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.521+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 shutdown_connections 2024-09-11T00:31:57.525 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:31:57.522+0000 7fa3ba800640 1 -- 172.21.15.103:0/2752620687 wait complete. 2024-09-11T00:31:57.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:57 smithi183 ceph-mon[35934]: pgmap v667: 129 pgs: 129 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-09-11T00:31:57.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:57 smithi103 ceph-mon[30578]: pgmap v667: 129 pgs: 129 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-09-11T00:31:57.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:57 smithi117 ceph-mon[35525]: pgmap v667: 129 pgs: 129 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-09-11T00:31:58.847 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '19d99768', quiesce took 6.6 seconds. Will release after: 21.799999999999997 2024-09-11T00:31:59.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:31:59 smithi103 ceph-mon[30578]: pgmap v668: 129 pgs: 129 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.0 MiB/s wr, 430 op/s 2024-09-11T00:31:59.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:31:59 smithi117 ceph-mon[35525]: pgmap v668: 129 pgs: 129 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.0 MiB/s wr, 430 op/s 2024-09-11T00:32:00.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:31:59 smithi183 ceph-mon[35934]: pgmap v668: 129 pgs: 129 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.0 MiB/s wr, 430 op/s 2024-09-11T00:32:01.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:01 smithi103 ceph-mon[30578]: pgmap v669: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 939 KiB/s wr, 373 op/s 2024-09-11T00:32:01.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:01 smithi117 ceph-mon[35525]: pgmap v669: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 939 KiB/s wr, 373 op/s 2024-09-11T00:32:02.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:01 smithi183 ceph-mon[35934]: pgmap v669: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 939 KiB/s wr, 373 op/s 2024-09-11T00:32:03.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:03 smithi103 ceph-mon[30578]: pgmap v670: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 783 KiB/s wr, 321 op/s 2024-09-11T00:32:03.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:03 smithi117 ceph-mon[35525]: pgmap v670: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 783 KiB/s wr, 321 op/s 2024-09-11T00:32:04.032 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:03 smithi183 ceph-mon[35934]: pgmap v670: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 783 KiB/s wr, 321 op/s 2024-09-11T00:32:05.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:05 smithi103 ceph-mon[30578]: pgmap v671: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 760 KiB/s wr, 283 op/s 2024-09-11T00:32:05.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:05 smithi117 ceph-mon[35525]: pgmap v671: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 760 KiB/s wr, 283 op/s 2024-09-11T00:32:06.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:05 smithi183 ceph-mon[35934]: pgmap v671: 129 pgs: 129 active+clean; 87 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 760 KiB/s wr, 283 op/s 2024-09-11T00:32:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 -- 172.21.15.103:0/1073157428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80102990 msgr2=0x7fed80102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 --2- 172.21.15.103:0/1073157428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80102990 0x7fed80102d90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fed74009930 tx=0x7fed7402a420 comp rx=0 tx=0).stop 2024-09-11T00:32:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 -- 172.21.15.103:0/1073157428 shutdown_connections 2024-09-11T00:32:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 --2- 172.21.15.103:0/1073157428 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed801044a0 0x7fed8010ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 --2- 172.21.15.103:0/1073157428 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80103ae0 0x7fed80103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 --2- 172.21.15.103:0/1073157428 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80102990 0x7fed80102d90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 -- 172.21.15.103:0/1073157428 >> 172.21.15.103:0/1073157428 conn(0x7fed800fe250 msgr2=0x7fed80100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 -- 172.21.15.103:0/1073157428 shutdown_connections 2024-09-11T00:32:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.149+0000 7fed87800640 1 -- 172.21.15.103:0/1073157428 wait complete. 2024-09-11T00:32:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.150+0000 7fed87800640 1 Processor -- start 2024-09-11T00:32:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.150+0000 7fed87800640 1 -- start start 2024-09-11T00:32:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed87800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80102990 0x7fed80198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed87800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80103ae0 0x7fed801993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed87800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed801044a0 0x7fed8019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed87800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed8010e600 con 0x7fed80103ae0 2024-09-11T00:32:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed87800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed8010e480 con 0x7fed80102990 2024-09-11T00:32:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed87800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed8010e780 con 0x7fed801044a0 2024-09-11T00:32:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80103ae0 0x7fed801993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80103ae0 0x7fed801993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36014/0 (socket says 172.21.15.103:36014) 2024-09-11T00:32:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed84e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80102990 0x7fed80198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 -- 172.21.15.103:0/2928028000 learned_addr learned my addr 172.21.15.103:0/2928028000 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed85800640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed801044a0 0x7fed8019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 -- 172.21.15.103:0/2928028000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed801044a0 msgr2=0x7fed8019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed801044a0 0x7fed8019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 -- 172.21.15.103:0/2928028000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80102990 msgr2=0x7fed80198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80102990 0x7fed80198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed7fe00640 1 -- 172.21.15.103:0/2928028000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8019a8c0 con 0x7fed80103ae0 2024-09-11T00:32:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.151+0000 7fed84e00640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80102990 0x7fed80198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.152+0000 7fed7fe00640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80103ae0 0x7fed801993d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e9f0 tx=0x7fed6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.152+0000 7fed7d600640 1 -- 172.21.15.103:0/2928028000 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c00cd80 con 0x7fed80103ae0 2024-09-11T00:32:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.152+0000 7fed7d600640 1 -- 172.21.15.103:0/2928028000 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed6c0108e0 con 0x7fed80103ae0 2024-09-11T00:32:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.152+0000 7fed7d600640 1 -- 172.21.15.103:0/2928028000 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c018980 con 0x7fed80103ae0 2024-09-11T00:32:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.152+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8019abb0 con 0x7fed80103ae0 2024-09-11T00:32:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.152+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed801a1280 con 0x7fed80103ae0 2024-09-11T00:32:06.169 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.155+0000 7fed8a7c7740 1 Processor -- start 2024-09-11T00:32:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.155+0000 7fed8a7c7740 1 -- start start 2024-09-11T00:32:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed7d600640 1 -- 172.21.15.103:0/2928028000 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fed6c010430 con 0x7fed80103ae0 2024-09-11T00:32:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8dcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8df8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562add8496b0 con 0x562add8dd4d0 2024-09-11T00:32:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562add838ae0 con 0x562add8daba0 2024-09-11T00:32:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562add83c8f0 con 0x562add8dff10 2024-09-11T00:32:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed84e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8df8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed84e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8df8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36020/0 (socket says 172.21.15.103:36020) 2024-09-11T00:32:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed84e00640 1 -- 172.21.15.103:0/2614856770 learned_addr learned my addr 172.21.15.103:0/2614856770 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed85800640 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8dcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.156+0000 7fed7fe00640 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed85800640 1 -- 172.21.15.103:0/2614856770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 msgr2=0x562add8e23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed85800640 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8e23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed85800640 1 -- 172.21.15.103:0/2614856770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 msgr2=0x562add8df8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed85800640 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8df8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed85800640 1 -- 172.21.15.103:0/2614856770 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562add805780 con 0x562add8daba0 2024-09-11T00:32:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed84e00640 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8df8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed85800640 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8dcf90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fed78005fd0 tx=0x7fed78011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509292224b273eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed72000640 1 -- 172.21.15.103:0/2614856770 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78004d20 con 0x562add8daba0 2024-09-11T00:32:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed72000640 1 -- 172.21.15.103:0/2614856770 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed78004030 con 0x562add8daba0 2024-09-11T00:32:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed72000640 1 -- 172.21.15.103:0/2614856770 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78005280 con 0x562add8daba0 2024-09-11T00:32:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed7d600640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fed680b52e0 0x7fed680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed84e00640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fed680b52e0 0x7fed680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.157+0000 7fed7d600640 1 -- 172.21.15.103:0/2928028000 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fed6c0e62b0 con 0x7fed80103ae0 2024-09-11T00:32:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2614856770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 msgr2=0x562add8dcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8dcf90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fed78005fd0 tx=0x7fed78011810 comp rx=0 tx=0).stop 2024-09-11T00:32:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2614856770 shutdown_connections 2024-09-11T00:32:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8e23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8df8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2614856770 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8dcf90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed84e00640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fed680b52e0 0x7fed680b77a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fed74000c00 tx=0x7fed740314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2614856770 >> 172.21.15.103:0/2614856770 conn(0x562add8d18e0 msgr2=0x562add8d2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2614856770 shutdown_connections 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.158+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2614856770 wait complete. 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 -- start start 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 Processor -- start 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8daba0 0x562add8e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8dd4d0 0x562add8e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dff10 0x562add8e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.175 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562add770ab0 con 0x562add8dff10 2024-09-11T00:32:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x562add7df570 con 0x562add8dd4d0 2024-09-11T00:32:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562add8e2ef0 con 0x562add8daba0 2024-09-11T00:32:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.159+0000 7fed8a7c7740 10 client.0 _ll_register_callbacks cb 0x562add7ee100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed84e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8daba0 0x562add8e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed85800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dff10 0x562add8e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed84e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8daba0 0x562add8e4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:58846/0 (socket says 172.21.15.103:58846) 2024-09-11T00:32:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed84e00640 1 -- 172.21.15.103:0/4033093169 learned_addr learned my addr 172.21.15.103:0/4033093169 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed7fe00640 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8dd4d0 0x562add8e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed85800640 1 -- 172.21.15.103:0/4033093169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8daba0 msgr2=0x562add8e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed85800640 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8daba0 0x562add8e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed85800640 1 -- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8dd4d0 msgr2=0x562add8e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed85800640 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8dd4d0 0x562add8e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed85800640 1 -- 172.21.15.103:0/4033093169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562add59a790 con 0x562add8dff10 2024-09-11T00:32:06.177 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed7fe00640 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8dd4d0 0x562add8e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed85800640 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dff10 0x562add8e55b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed78011d10 tx=0x7fed78004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f74a21086a6504e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.160+0000 7fed66a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed66a00640 1 -- 172.21.15.103:0/4033093169 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78027020 con 0x562add8dff10 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed78004030 con 0x562add8dff10 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed66a00640 1 -- 172.21.15.103:0/4033093169 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fed78003a30 con 0x562add8dff10 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed66a00640 1 -- 172.21.15.103:0/4033093169 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fed78032050 con 0x562add8dff10 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 10 client.16272 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562add613620 con 0x562add8dff10 2024-09-11T00:32:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed66a00640 1 -- 172.21.15.103:0/4033093169 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed78004030 con 0x562add8dff10 2024-09-11T00:32:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed66a00640 1 -- 172.21.15.103:0/4033093169 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fed78018070 con 0x562add8dff10 2024-09-11T00:32:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 10 client.16272 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 20 client.16272 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add8efa00 0x562add8f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 4 client.16272 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.161+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562add84b600 con 0x562add8efa00 2024-09-11T00:32:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.162+0000 7fed84e00640 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add8efa00 0x562add8f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.162+0000 7fed84e00640 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add8efa00 0x562add8f1df0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3bb06b3e8ad0df server_cookie=31178002b74cfde0 in_seq=0 out_seq=0 2024-09-11T00:32:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.162+0000 7fed66a00640 10 client.16272 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.163+0000 7fed66a00640 1 -- 172.21.15.103:0/4033093169 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562add84b600 con 0x562add8efa00 2024-09-11T00:32:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.163+0000 7fed66a00640 10 client.16272 handle_command_reply: tid=0 2024-09-11T00:32:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.163+0000 7fed8a7c7740 1 client.16272 shutdown 2024-09-11T00:32:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dff10 msgr2=0x562add8e55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dff10 0x562add8e55b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed78011d10 tx=0x7fed78004ac0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 shutdown_connections 2024-09-11T00:32:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dff10 0x562add8e55b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8dd4d0 0x562add8e49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add8efa00 0x562add8f1df0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/4033093169 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8daba0 0x562add8e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed66a00640 0 client.16272 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 >> 172.21.15.103:0/4033093169 conn(0x562add8d18e0 msgr2=0x562add8d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 shutdown_connections 2024-09-11T00:32:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 1 -- 172.21.15.103:0/4033093169 wait complete. 2024-09-11T00:32:06.182 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.164+0000 7fed8a7c7740 20 client.16272 trim_cache size 0 max 16384 2024-09-11T00:32:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 Processor -- start 2024-09-11T00:32:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 -- start start 2024-09-11T00:32:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed801abd40 con 0x562add8dd4d0 2024-09-11T00:32:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a9610 con 0x562add8daba0 2024-09-11T00:32:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.179+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a7460 con 0x562add8dff10 2024-09-11T00:32:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed84e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8d3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36036/0 (socket says 172.21.15.103:36036) 2024-09-11T00:32:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed7fe00640 1 -- 172.21.15.103:0/2898555589 learned_addr learned my addr 172.21.15.103:0/2898555589 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed85800640 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed84e00640 1 -- 172.21.15.103:0/2898555589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 msgr2=0x562add8d3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed84e00640 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8d3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed84e00640 1 -- 172.21.15.103:0/2898555589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 msgr2=0x562add8d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed84e00640 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed84e00640 1 -- 172.21.15.103:0/2898555589 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562add8f2160 con 0x562add8daba0 2024-09-11T00:32:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed7fe00640 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.180+0000 7fed84e00640 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8d22e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x562add8f25f0 tx=0x7fed74040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a130374846c9ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed71600640 1 -- 172.21.15.103:0/2898555589 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74035040 con 0x562add8daba0 2024-09-11T00:32:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed71600640 1 -- 172.21.15.103:0/2898555589 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed74003b60 con 0x562add8daba0 2024-09-11T00:32:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed71600640 1 -- 172.21.15.103:0/2898555589 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7403db40 con 0x562add8daba0 2024-09-11T00:32:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2898555589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 msgr2=0x562add8d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8d22e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x562add8f25f0 tx=0x7fed74040000 comp rx=0 tx=0).stop 2024-09-11T00:32:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2898555589 shutdown_connections 2024-09-11T00:32:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8d3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/2898555589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8d22e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2898555589 >> 172.21.15.103:0/2898555589 conn(0x562add8d18e0 msgr2=0x562add8e02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2898555589 shutdown_connections 2024-09-11T00:32:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.181+0000 7fed8a7c7740 1 -- 172.21.15.103:0/2898555589 wait complete. 2024-09-11T00:32:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 -- start start 2024-09-11T00:32:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 Processor -- start 2024-09-11T00:32:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8ea9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8eb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a7ad0 con 0x562add8dd4d0 2024-09-11T00:32:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a8f40 con 0x562add8daba0 2024-09-11T00:32:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed801a81a0 con 0x562add8dff10 2024-09-11T00:32:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed8a7c7740 10 client.0 _ll_register_callbacks cb 0x7fed780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8ea9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed84e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed7fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8ea9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36038/0 (socket says 172.21.15.103:36038) 2024-09-11T00:32:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed84e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8e8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41588/0 (socket says 172.21.15.103:41588) 2024-09-11T00:32:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.182+0000 7fed7fe00640 1 -- 172.21.15.103:0/3038476257 learned_addr learned my addr 172.21.15.103:0/3038476257 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed85800640 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed84e00640 1 -- 172.21.15.103:0/3038476257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 msgr2=0x562add8eb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed84e00640 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8eb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed84e00640 1 -- 172.21.15.103:0/3038476257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 msgr2=0x562add8ea9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed84e00640 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8ea9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed84e00640 1 -- 172.21.15.103:0/3038476257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562add8dd300 con 0x562add8daba0 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed7fe00640 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8ea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed84e00640 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8e8460 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed74004650 tx=0x7fed7403d4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5235f1dede5e6263 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed66a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:32:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed66a00640 1 -- 172.21.15.103:0/3038476257 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74068060 con 0x562add8daba0 2024-09-11T00:32:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed74003b60 con 0x562add8daba0 2024-09-11T00:32:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.183+0000 7fed66a00640 1 -- 172.21.15.103:0/3038476257 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fed7405f040 con 0x562add8daba0 2024-09-11T00:32:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed66a00640 1 -- 172.21.15.103:0/3038476257 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fed74035040 con 0x562add8daba0 2024-09-11T00:32:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 10 client.26345 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562add961470 con 0x562add8daba0 2024-09-11T00:32:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed66a00640 1 -- 172.21.15.103:0/3038476257 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed74003b60 con 0x562add8daba0 2024-09-11T00:32:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed66a00640 1 -- 172.21.15.103:0/3038476257 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fed74044040 con 0x562add8daba0 2024-09-11T00:32:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 10 client.26345 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 10 client.26345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 20 client.26345 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add961dd0 0x562add9641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 4 client.26345 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:32:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.184+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562add8470b0 con 0x562add961dd0 2024-09-11T00:32:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.185+0000 7fed7fe00640 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add961dd0 0x562add9641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.185+0000 7fed66a00640 10 client.26345 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.185+0000 7fed7fe00640 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add961dd0 0x562add9641f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68f17a9de7b4b88 server_cookie=1e866edd1d97c57e in_seq=0 out_seq=0 2024-09-11T00:32:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.198+0000 7fed66a00640 1 -- 172.21.15.103:0/3038476257 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562add8470b0 con 0x562add961dd0 2024-09-11T00:32:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.198+0000 7fed66a00640 10 client.26345 handle_command_reply: tid=0 2024-09-11T00:32:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.198+0000 7fed8a7c7740 1 client.26345 shutdown 2024-09-11T00:32:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 msgr2=0x562add8e8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8e8460 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed74004650 tx=0x7fed7403d4e0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 shutdown_connections 2024-09-11T00:32:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x562add961dd0 0x562add9641f0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562add8dff10 0x562add8eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562add8dd4d0 0x562add8ea9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 --2- 172.21.15.103:0/3038476257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x562add8daba0 0x562add8e8460 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed66a00640 0 client.26345 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 >> 172.21.15.103:0/3038476257 conn(0x562add8d18e0 msgr2=0x562add8e02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 shutdown_connections 2024-09-11T00:32:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 1 -- 172.21.15.103:0/3038476257 wait complete. 2024-09-11T00:32:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.199+0000 7fed8a7c7740 20 client.26345 trim_cache size 0 max 16384 2024-09-11T00:32:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:32:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:32:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:32:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:32:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fed680b52e0 msgr2=0x7fed680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fed680b52e0 0x7fed680b77a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fed74000c00 tx=0x7fed740314a0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80103ae0 msgr2=0x7fed801993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80103ae0 0x7fed801993d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00e9f0 tx=0x7fed6c00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 shutdown_connections 2024-09-11T00:32:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fed680b52e0 0x7fed680b77a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed801044a0 0x7fed8019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed80103ae0 0x7fed801993d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 --2- 172.21.15.103:0/2928028000 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed80102990 0x7fed80198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 >> 172.21.15.103:0/2928028000 conn(0x7fed800fe250 msgr2=0x7fed80106ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 shutdown_connections 2024-09-11T00:32:06.210 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:06.200+0000 7fed87800640 1 -- 172.21.15.103:0/2928028000 wait complete. 2024-09-11T00:32:07.921 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:07 smithi103 ceph-mon[30578]: pgmap v672: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 486 KiB/s wr, 158 op/s 2024-09-11T00:32:07.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:07 smithi117 ceph-mon[35525]: pgmap v672: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 486 KiB/s wr, 158 op/s 2024-09-11T00:32:08.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:07 smithi183 ceph-mon[35934]: pgmap v672: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 486 KiB/s wr, 158 op/s 2024-09-11T00:32:09.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:09 smithi103 ceph-mon[30578]: pgmap v673: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 380 KiB/s wr, 123 op/s 2024-09-11T00:32:09.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:09 smithi117 ceph-mon[35525]: pgmap v673: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 380 KiB/s wr, 123 op/s 2024-09-11T00:32:10.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:09 smithi183 ceph-mon[35934]: pgmap v673: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 380 KiB/s wr, 123 op/s 2024-09-11T00:32:11.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:11 smithi103 ceph-mon[30578]: pgmap v674: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 248 KiB/s wr, 65 op/s 2024-09-11T00:32:11.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:11 smithi117 ceph-mon[35525]: pgmap v674: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 248 KiB/s wr, 65 op/s 2024-09-11T00:32:12.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:11 smithi183 ceph-mon[35934]: pgmap v674: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 248 KiB/s wr, 65 op/s 2024-09-11T00:32:13.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:13 smithi103 ceph-mon[30578]: pgmap v675: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 129 KiB/s wr, 24 op/s 2024-09-11T00:32:13.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:13 smithi117 ceph-mon[35525]: pgmap v675: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 129 KiB/s wr, 24 op/s 2024-09-11T00:32:14.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:13 smithi183 ceph-mon[35934]: pgmap v675: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 129 KiB/s wr, 24 op/s 2024-09-11T00:32:15.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:15 smithi103 ceph-mon[30578]: pgmap v676: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 100 KiB/s wr, 11 op/s 2024-09-11T00:32:15.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:15 smithi117 ceph-mon[35525]: pgmap v676: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 100 KiB/s wr, 11 op/s 2024-09-11T00:32:16.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:15 smithi183 ceph-mon[35934]: pgmap v676: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 100 KiB/s wr, 11 op/s 2024-09-11T00:32:17.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:17 smithi103 ceph-mon[30578]: pgmap v677: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-11T00:32:17.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:17 smithi117 ceph-mon[35525]: pgmap v677: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-11T00:32:18.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:17 smithi183 ceph-mon[35934]: pgmap v677: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.325+0000 7fc82ea00640 1 -- 172.21.15.103:0/3789916564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 msgr2=0x7fc828104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.325+0000 7fc82ea00640 1 --2- 172.21.15.103:0/3789916564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 0x7fc828104f90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc818009be0 tx=0x7fc81802a6c0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.326+0000 7fc82ea00640 1 -- 172.21.15.103:0/3789916564 shutdown_connections 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.326+0000 7fc82ea00640 1 --2- 172.21.15.103:0/3789916564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8281066a0 0x7fc82810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.326+0000 7fc82ea00640 1 --2- 172.21.15.103:0/3789916564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc828105ce0 0x7fc828106160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.326+0000 7fc82ea00640 1 --2- 172.21.15.103:0/3789916564 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 0x7fc828104f90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.326+0000 7fc82ea00640 1 -- 172.21.15.103:0/3789916564 >> 172.21.15.103:0/3789916564 conn(0x7fc828100470 msgr2=0x7fc828102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.326+0000 7fc82ea00640 1 -- 172.21.15.103:0/3789916564 shutdown_connections 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.326+0000 7fc82ea00640 1 -- 172.21.15.103:0/3789916564 wait complete. 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 Processor -- start 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 -- start start 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 0x7fc82819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc828105ce0 0x7fc82819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8281066a0 0x7fc82819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc828110800 con 0x7fc828105ce0 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc828110680 con 0x7fc828104b90 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ea00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc828110980 con 0x7fc8281066a0 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc81f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc828105ce0 0x7fc82819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc81f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc828105ce0 0x7fc82819b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37442/0 (socket says 172.21.15.103:37442) 2024-09-11T00:32:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc81f400640 1 -- 172.21.15.103:0/882813309 learned_addr learned my addr 172.21.15.103:0/882813309 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc827e00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 0x7fc82819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.327+0000 7fc82ca00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8281066a0 0x7fc82819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc81f400640 1 -- 172.21.15.103:0/882813309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8281066a0 msgr2=0x7fc82819c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc81f400640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8281066a0 0x7fc82819c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc81f400640 1 -- 172.21.15.103:0/882813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 msgr2=0x7fc82819af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc81f400640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 0x7fc82819af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc81f400640 1 -- 172.21.15.103:0/882813309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82819cac0 con 0x7fc828105ce0 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc827e00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 0x7fc82819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc81f400640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc828105ce0 0x7fc82819b5d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e9f0 tx=0x7fc81000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc825600640 1 -- 172.21.15.103:0/882813309 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81000cd80 con 0x7fc828105ce0 2024-09-11T00:32:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc825600640 1 -- 172.21.15.103:0/882813309 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8100054d0 con 0x7fc828105ce0 2024-09-11T00:32:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82819cdb0 con 0x7fc828105ce0 2024-09-11T00:32:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.328+0000 7fc825600640 1 -- 172.21.15.103:0/882813309 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc810010690 con 0x7fc828105ce0 2024-09-11T00:32:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.329+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8281a3480 con 0x7fc828105ce0 2024-09-11T00:32:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.331+0000 7fc8317e2740 1 Processor -- start 2024-09-11T00:32:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc8317e2740 1 -- start start 2024-09-11T00:32:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc825600640 1 -- 172.21.15.103:0/882813309 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fc810004e90 con 0x7fc828105ce0 2024-09-11T00:32:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44691eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f4469214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f446921b00 0x55f446923fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc8317e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f44688f6b0 con 0x55f446921b00 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc8317e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f44687eae0 con 0x55f44691f0c0 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc8317e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f4468828f0 con 0x55f44691c790 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc81f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f446921b00 0x55f446923fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc81f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f446921b00 0x55f446923fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37456/0 (socket says 172.21.15.103:37456) 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc81f400640 1 -- 172.21.15.103:0/1936506967 learned_addr learned my addr 172.21.15.103:0/1936506967 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc827e00640 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f4469214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.332+0000 7fc82ca00640 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44691eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81f400640 1 -- 172.21.15.103:0/1936506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 msgr2=0x55f44691eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81f400640 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44691eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81f400640 1 -- 172.21.15.103:0/1936506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 msgr2=0x55f4469214b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81f400640 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f4469214b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81f400640 1 -- 172.21.15.103:0/1936506967 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44684b780 con 0x55f446921b00 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81f400640 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f446921b00 0x55f446923fc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8100ee3c0 tx=0x7fc8100ee4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46160f558c72237 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81d200640 1 -- 172.21.15.103:0/1936506967 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8100e8310 con 0x55f446921b00 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81d200640 1 -- 172.21.15.103:0/1936506967 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8100e84b0 con 0x55f446921b00 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.333+0000 7fc81d200640 1 -- 172.21.15.103:0/1936506967 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8100f5970 con 0x55f446921b00 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc825600640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc80c0b52e0 0x7fc80c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc825600640 1 -- 172.21.15.103:0/882813309 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc8100e63d0 con 0x7fc828105ce0 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 -- 172.21.15.103:0/1936506967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f446921b00 msgr2=0x55f446923fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f446921b00 0x55f446923fc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc8100ee3c0 tx=0x7fc8100ee4b0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc82ca00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc80c0b52e0 0x7fc80c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 -- 172.21.15.103:0/1936506967 shutdown_connections 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f446921b00 0x55f446923fc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f4469214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 --2- 172.21.15.103:0/1936506967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44691eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 -- 172.21.15.103:0/1936506967 >> 172.21.15.103:0/1936506967 conn(0x55f4469178e0 msgr2=0x55f446918670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 -- 172.21.15.103:0/1936506967 shutdown_connections 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.334+0000 7fc8317e2740 1 -- 172.21.15.103:0/1936506967 wait complete. 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc82ca00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc80c0b52e0 0x7fc80c0b77a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc82819bd00 tx=0x7fc820009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 -- start start 2024-09-11T00:32:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 Processor -- start 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446925ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f446926590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f446921b00 0x55f446927160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4467b6ab0 con 0x55f44691c790 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55f446825570 con 0x55f44691f0c0 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f446924ae0 con 0x55f446921b00 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.335+0000 7fc8317e2740 10 client.0 _ll_register_callbacks cb 0x55f446834100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446925ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446925ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37472/0 (socket says 172.21.15.103:37472) 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc81f400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f446926590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 -- 172.21.15.103:0/904741387 learned_addr learned my addr 172.21.15.103:0/904741387 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc82ca00640 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f446921b00 0x55f446927160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 -- 172.21.15.103:0/904741387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f446921b00 msgr2=0x55f446927160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f446921b00 0x55f446927160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 -- 172.21.15.103:0/904741387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 msgr2=0x55f446926590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f446926590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 -- 172.21.15.103:0/904741387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4465e0790 con 0x55f44691c790 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc827e00640 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446925ef0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc818006010 tx=0x7fc818030dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62dfde53237c6aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.336+0000 7fc815600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc815600640 1 -- 172.21.15.103:0/904741387 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818036040 con 0x55f44691c790 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc815600640 1 -- 172.21.15.103:0/904741387 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc818003b70 con 0x55f44691c790 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8100e84b0 con 0x55f44691c790 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc815600640 1 -- 172.21.15.103:0/904741387 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc818050050 con 0x55f44691c790 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc8317e2740 10 client.16290 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f446659620 con 0x55f44691c790 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc815600640 1 -- 172.21.15.103:0/904741387 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8100e84b0 con 0x55f44691c790 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc815600640 1 -- 172.21.15.103:0/904741387 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fc818031440 con 0x55f44691c790 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc8317e2740 10 client.16290 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc8317e2740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.337+0000 7fc8317e2740 20 client.16290 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.338+0000 7fc8317e2740 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f4469317d0 0x55f446933c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.338+0000 7fc8317e2740 4 client.16290 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.338+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f446891600 con 0x55f4469317d0 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.338+0000 7fc81f400640 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f4469317d0 0x55f446933c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.338+0000 7fc81f400640 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f4469317d0 0x55f446933c30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6899cceaf090fdc server_cookie=a33c2beef9e5454 in_seq=0 out_seq=0 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.338+0000 7fc815600640 10 client.16290 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.339+0000 7fc815600640 1 -- 172.21.15.103:0/904741387 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f446891600 con 0x55f4469317d0 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.339+0000 7fc815600640 10 client.16290 handle_command_reply: tid=0 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.339+0000 7fc8317e2740 1 client.16290 shutdown 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.339+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 msgr2=0x55f446925ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.339+0000 7fc8317e2740 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446925ef0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc818006010 tx=0x7fc818030dd0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 shutdown_connections 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f4469317d0 0x55f446933c30 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f446921b00 0x55f446927160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc815600640 0 client.16290 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f44691f0c0 0x55f446926590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 --2- 172.21.15.103:0/904741387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446925ef0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 >> 172.21.15.103:0/904741387 conn(0x55f4469178e0 msgr2=0x55f446916a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 shutdown_connections 2024-09-11T00:32:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 1 -- 172.21.15.103:0/904741387 wait complete. 2024-09-11T00:32:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.340+0000 7fc8317e2740 20 client.16290 trim_cache size 0 max 16384 2024-09-11T00:32:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 Processor -- start 2024-09-11T00:32:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 -- start start 2024-09-11T00:32:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446924ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691f0c0 0x55f446925420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f446919710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281b0070 con 0x55f44691c790 2024-09-11T00:32:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ad940 con 0x55f446921b00 2024-09-11T00:32:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc8317e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ab790 con 0x55f44691f0c0 2024-09-11T00:32:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc827e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446924ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc827e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446924ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37478/0 (socket says 172.21.15.103:37478) 2024-09-11T00:32:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc827e00640 1 -- 172.21.15.103:0/855708211 learned_addr learned my addr 172.21.15.103:0/855708211 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc81f400640 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691f0c0 0x55f446925420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.355+0000 7fc82ca00640 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f446919710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc827e00640 1 -- 172.21.15.103:0/855708211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691f0c0 msgr2=0x55f446925420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc827e00640 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691f0c0 0x55f446925420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc827e00640 1 -- 172.21.15.103:0/855708211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 msgr2=0x55f446919710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc827e00640 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f446919710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc827e00640 1 -- 172.21.15.103:0/855708211 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44691eef0 con 0x55f44691c790 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc82ca00640 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f446919710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc827e00640 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446924ee0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc818000f80 tx=0x7fc818030fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6800892e565ba2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc817e00640 1 -- 172.21.15.103:0/855708211 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818059070 con 0x55f44691c790 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc817e00640 1 -- 172.21.15.103:0/855708211 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818031a50 con 0x55f44691c790 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.356+0000 7fc817e00640 1 -- 172.21.15.103:0/855708211 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818036070 con 0x55f44691c790 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 -- 172.21.15.103:0/855708211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 msgr2=0x55f446924ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446924ee0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc818000f80 tx=0x7fc818030fd0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 -- 172.21.15.103:0/855708211 shutdown_connections 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f446919710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691f0c0 0x55f446925420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 --2- 172.21.15.103:0/855708211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691c790 0x55f446924ee0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 -- 172.21.15.103:0/855708211 >> 172.21.15.103:0/855708211 conn(0x55f4469178e0 msgr2=0x55f446918140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 -- 172.21.15.103:0/855708211 shutdown_connections 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 -- 172.21.15.103:0/855708211 wait complete. 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 -- start start 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.357+0000 7fc8317e2740 1 Processor -- start 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44692da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.366 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691f0c0 0x55f44692c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc8317e2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f44692cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc8317e2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281abe00 con 0x55f44691f0c0 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc8317e2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ad270 con 0x55f446921b00 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc8317e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc8281ac4d0 con 0x55f44691c790 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc8317e2740 10 client.0 _ll_register_callbacks cb 0x7fc810005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc81f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691f0c0 0x55f44692c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc81f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691f0c0 0x55f44692c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:37480/0 (socket says 172.21.15.103:37480) 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc81f400640 1 -- 172.21.15.103:0/3649785611 learned_addr learned my addr 172.21.15.103:0/3649785611 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc827e00640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44692da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc82ca00640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f44692cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc81f400640 1 -- 172.21.15.103:0/3649785611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 msgr2=0x55f44692da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.367 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc81f400640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44692da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc81f400640 1 -- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 msgr2=0x55f44692cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.358+0000 7fc81f400640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f44692cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc81f400640 1 -- 172.21.15.103:0/3649785611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f44692d4a0 con 0x55f44691f0c0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc82ca00640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f44692cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc81f400640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691f0c0 0x55f44692c710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc8100ac730 tx=0x7fc8100a8800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fd68c39fdd0996 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc815600640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc815600640 1 -- 172.21.15.103:0/3649785611 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8100a8df0 con 0x55f44691f0c0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc815600640 1 -- 172.21.15.103:0/3649785611 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fc8100e60d0 con 0x55f44691f0c0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc818031a50 con 0x55f44691f0c0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc815600640 1 -- 172.21.15.103:0/3649785611 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fc8100abf10 con 0x55f44691f0c0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc8317e2740 10 client.16296 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f44699e830 con 0x55f44691f0c0 2024-09-11T00:32:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.359+0000 7fc815600640 1 -- 172.21.15.103:0/3649785611 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc818031a50 con 0x55f44691f0c0 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc815600640 1 -- 172.21.15.103:0/3649785611 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fc8100051b0 con 0x55f44691f0c0 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc8317e2740 10 client.16296 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc8317e2740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc8317e2740 20 client.16296 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc8317e2740 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f44699ecc0 0x55f4469a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc8317e2740 4 client.16296 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f44688d0b0 con 0x55f44699ecc0 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.360+0000 7fc827e00640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f44699ecc0 0x55f4469a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.361+0000 7fc815600640 10 client.16296 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.361+0000 7fc827e00640 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f44699ecc0 0x55f4469a10e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6befa82ad688d93 server_cookie=bb52b424f819cf08 in_seq=0 out_seq=0 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.370+0000 7fc815600640 1 -- 172.21.15.103:0/3649785611 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f44688d0b0 con 0x55f44699ecc0 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc815600640 10 client.16296 handle_command_reply: tid=0 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 client.16296 shutdown 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691f0c0 msgr2=0x55f44692c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691f0c0 0x55f44692c710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc8100ac730 tx=0x7fc8100a8800 comp rx=0 tx=0).stop 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 shutdown_connections 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55f446921b00 0x55f44692cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f44691f0c0 0x55f44692c710 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55f44699ecc0 0x55f4469a10e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 --2- 172.21.15.103:0/3649785611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f44691c790 0x55f44692da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc815600640 0 client.16296 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 >> 172.21.15.103:0/3649785611 conn(0x55f4469178e0 msgr2=0x55f446918140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 shutdown_connections 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 1 -- 172.21.15.103:0/3649785611 wait complete. 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.371+0000 7fc8317e2740 20 client.16296 trim_cache size 0 max 16384 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:32:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:32:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc80c0b52e0 msgr2=0x7fc80c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc80c0b52e0 0x7fc80c0b77a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc82819bd00 tx=0x7fc820009260 comp rx=0 tx=0).stop 2024-09-11T00:32:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc828105ce0 msgr2=0x7fc82819b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc828105ce0 0x7fc82819b5d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc81000e9f0 tx=0x7fc81000edf0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 shutdown_connections 2024-09-11T00:32:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fc80c0b52e0 0x7fc80c0b77a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc8281066a0 0x7fc82819c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc828105ce0 0x7fc82819b5d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 --2- 172.21.15.103:0/882813309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc828104b90 0x7fc82819af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 >> 172.21.15.103:0/882813309 conn(0x7fc828100470 msgr2=0x7fc8281090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 shutdown_connections 2024-09-11T00:32:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:19.372+0000 7fc82ea00640 1 -- 172.21.15.103:0/882813309 wait complete. 2024-09-11T00:32:19.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:19 smithi103 ceph-mon[30578]: pgmap v678: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-11T00:32:19.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:19 smithi117 ceph-mon[35525]: pgmap v678: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-11T00:32:20.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:19 smithi183 ceph-mon[35934]: pgmap v678: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-09-11T00:32:20.649 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:32:21.036 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 -- 172.21.15.103:0/4218347728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10a820 msgr2=0x7fd30c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 --2- 172.21.15.103:0/4218347728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10a820 0x7fd30c10ac60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009900 tx=0x7fd2f802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 -- 172.21.15.103:0/4218347728 shutdown_connections 2024-09-11T00:32:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 --2- 172.21.15.103:0/4218347728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c10b1a0 0x7fd30c1119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.687 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 --2- 172.21.15.103:0/4218347728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10a820 0x7fd30c10ac60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 --2- 172.21.15.103:0/4218347728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd30c104e10 0x7fd30c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 -- 172.21.15.103:0/4218347728 >> 172.21.15.103:0/4218347728 conn(0x7fd30c100840 msgr2=0x7fd30c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 -- 172.21.15.103:0/4218347728 shutdown_connections 2024-09-11T00:32:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.684+0000 7fd312000640 1 -- 172.21.15.103:0/4218347728 wait complete. 2024-09-11T00:32:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 Processor -- start 2024-09-11T00:32:21.688 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 -- start start 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c104e10 0x7fd30c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd30c10a820 0x7fd30c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10b1a0 0x7fd30c1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c114e70 con 0x7fd30c10b1a0 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c114cf0 con 0x7fd30c104e10 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.685+0000 7fd312000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd30c114ff0 con 0x7fd30c10a820 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c104e10 0x7fd30c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c104e10 0x7fd30c19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:41150/0 (socket says 172.21.15.103:41150) 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 -- 172.21.15.103:0/3891340346 learned_addr learned my addr 172.21.15.103:0/3891340346 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd311600640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10b1a0 0x7fd30c1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.690 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd30be00640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd30c10a820 0x7fd30c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 -- 172.21.15.103:0/3891340346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd30c10a820 msgr2=0x7fd30c19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd30c10a820 0x7fd30c19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 -- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10b1a0 msgr2=0x7fd30c1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10b1a0 0x7fd30c1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 -- 172.21.15.103:0/3891340346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30c1a1350 con 0x7fd30c104e10 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.686+0000 7fd310c00640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c104e10 0x7fd30c19b2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd30000e9f0 tx=0x7fd30000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.687+0000 7fd309600640 1 -- 172.21.15.103:0/3891340346 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30000cd80 con 0x7fd30c104e10 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.687+0000 7fd309600640 1 -- 172.21.15.103:0/3891340346 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3000054d0 con 0x7fd30c104e10 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.687+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30c1a1640 con 0x7fd30c104e10 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.687+0000 7fd309600640 1 -- 172.21.15.103:0/3891340346 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300010690 con 0x7fd30c104e10 2024-09-11T00:32:21.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.687+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30c1a7d00 con 0x7fd30c104e10 2024-09-11T00:32:21.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.688+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd30c1051f0 con 0x7fd30c104e10 2024-09-11T00:32:21.692 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.690+0000 7fd309600640 1 -- 172.21.15.103:0/3891340346 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd300004e90 con 0x7fd30c104e10 2024-09-11T00:32:21.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.691+0000 7fd309600640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd2f40b5410 0x7fd2f40b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:21.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.692+0000 7fd309600640 1 -- 172.21.15.103:0/3891340346 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd3000e6350 con 0x7fd30c104e10 2024-09-11T00:32:21.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.692+0000 7fd30be00640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd2f40b5410 0x7fd2f40b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:21.694 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.692+0000 7fd30be00640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd2f40b5410 0x7fd2f40b78d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005ab0 tx=0x7fd2f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:21.698 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.696+0000 7fd309600640 1 -- 172.21.15.103:0/3891340346 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd3000ace20 con 0x7fd30c104e10 2024-09-11T00:32:21.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.916+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd30c02e5c0 con 0x7fd30c104e10 2024-09-11T00:32:21.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:21 smithi103 ceph-mon[30578]: pgmap v679: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-11T00:32:21.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.917+0000 7fd309600640 1 -- 172.21.15.103:0/3891340346 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11933 (secure 0 0 0) 0x7fd3000a6e20 con 0x7fd30c104e10 2024-09-11T00:32:21.920 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:32:21.920 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":44,"btime":"2024-09-11T00:31:53:238018+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:31:53.238002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26323":{"gid":26323,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4278110475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4278110475},{"type":"v1","addr":"172.21.15.183:6835","nonce":4278110475}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24656,24643,24628]},"id":1}]} 2024-09-11T00:32:21.920 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 44 2024-09-11T00:32:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd2f40b5410 msgr2=0x7fd2f40b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd2f40b5410 0x7fd2f40b78d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8005ab0 tx=0x7fd2f8005a20 comp rx=0 tx=0).stop 2024-09-11T00:32:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c104e10 msgr2=0x7fd30c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c104e10 0x7fd30c19b2a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd30000e9f0 tx=0x7fd30000edf0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 shutdown_connections 2024-09-11T00:32:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd2f40b5410 0x7fd2f40b78d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd30c10b1a0 0x7fd30c1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd30c10a820 0x7fd30c19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 --2- 172.21.15.103:0/3891340346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd30c104e10 0x7fd30c19b2a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 >> 172.21.15.103:0/3891340346 conn(0x7fd30c100840 msgr2=0x7fd30c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 shutdown_connections 2024-09-11T00:32:21.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:21.920+0000 7fd312000640 1 -- 172.21.15.103:0/3891340346 wait complete. 2024-09-11T00:32:21.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:21 smithi117 ceph-mon[35525]: pgmap v679: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-11T00:32:22.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:21 smithi183 ceph-mon[35934]: pgmap v679: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-11T00:32:22.735 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --set-id 19d99768 --release --await' 2024-09-11T00:32:22.735 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --set-id 19d99768 --release --await 2024-09-11T00:32:22.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:22 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3891340346' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:22.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:22 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3891340346' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:23.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:22 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3891340346' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:23.106 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:23.743 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.711+0000 7fe2c2000640 1 -- 172.21.15.103:0/3935650405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc077ff0 msgr2=0x7fe2bc1132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.711+0000 7fe2c2000640 1 --2- 172.21.15.103:0/3935650405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc077ff0 0x7fe2bc1132c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009900 tx=0x7fe2a802a3f0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.711+0000 7fe2c2000640 1 -- 172.21.15.103:0/3935650405 shutdown_connections 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.711+0000 7fe2c2000640 1 --2- 172.21.15.103:0/3935650405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc113800 0x7fe2bc115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.711+0000 7fe2c2000640 1 --2- 172.21.15.103:0/3935650405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc077ff0 0x7fe2bc1132c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.711+0000 7fe2c2000640 1 --2- 172.21.15.103:0/3935650405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 0x7fe2bc077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.711+0000 7fe2c2000640 1 -- 172.21.15.103:0/3935650405 >> 172.21.15.103:0/3935650405 conn(0x7fe2bc100840 msgr2=0x7fe2bc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.712+0000 7fe2c2000640 1 -- 172.21.15.103:0/3935650405 shutdown_connections 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.712+0000 7fe2c2000640 1 -- 172.21.15.103:0/3935650405 wait complete. 2024-09-11T00:32:23.744 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 Processor -- start 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 -- start start 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 0x7fe2bc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc077ff0 0x7fe2bc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc113800 0x7fe2bc1a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1191e0 con 0x7fe2bc113800 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc119060 con 0x7fe2bc077ff0 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.713+0000 7fe2c2000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc119360 con 0x7fe2bc0776d0 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 0x7fe2bc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc113800 0x7fe2bc1a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 0x7fe2bc19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59430/0 (socket says 172.21.15.103:59430) 2024-09-11T00:32:23.745 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 -- 172.21.15.103:0/1204450563 learned_addr learned my addr 172.21.15.103:0/1204450563 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc077ff0 0x7fe2bc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 -- 172.21.15.103:0/1204450563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc077ff0 msgr2=0x7fe2bc19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc077ff0 0x7fe2bc19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 -- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc113800 msgr2=0x7fe2bc1a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc113800 0x7fe2bc1a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.714+0000 7fe2c0c00640 1 -- 172.21.15.103:0/1204450563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2bc1a5760 con 0x7fe2bc0776d0 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.715+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 0x7fe2bc19f6b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e9f0 tx=0x7fe2b000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.715+0000 7fe2b9600640 1 -- 172.21.15.103:0/1204450563 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000ccc0 con 0x7fe2bc0776d0 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.715+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2bc1a5a50 con 0x7fe2bc0776d0 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.715+0000 7fe2b9600640 1 -- 172.21.15.103:0/1204450563 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b00108e0 con 0x7fe2bc0776d0 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.715+0000 7fe2b9600640 1 -- 172.21.15.103:0/1204450563 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0018880 con 0x7fe2bc0776d0 2024-09-11T00:32:23.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.715+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2bc1ac2e0 con 0x7fe2bc0776d0 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.718+0000 7fe2c61c3740 1 Processor -- start 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.718+0000 7fe2c61c3740 1 -- start start 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.718+0000 7fe2b9600640 1 -- 172.21.15.103:0/1204450563 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe2b0010430 con 0x7fe2bc0776d0 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715cbaf0 0x5588715d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715d7a80 0x5588715d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558871542290 con 0x5588715cb1d0 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588715316c0 con 0x5588715cbaf0 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5588715354d0 con 0x5588715d7a80 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715cbaf0 0x5588715d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40706/0 (socket says 172.21.15.103:40706) 2024-09-11T00:32:23.747 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 -- 172.21.15.103:0/2969919042 learned_addr learned my addr 172.21.15.103:0/2969919042 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715d7a80 0x5588715d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 -- 172.21.15.103:0/2969919042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715d7a80 msgr2=0x5588715d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715d7a80 0x5588715d9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 -- 172.21.15.103:0/2969919042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715cbaf0 msgr2=0x5588715d7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715cbaf0 0x5588715d7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.719+0000 7fe2c1600640 1 -- 172.21.15.103:0/2969919042 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588714be460 con 0x5588715cb1d0 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.720+0000 7fe2c1600640 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715cb5b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fd0 tx=0x7fe2b40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ea1d1c02930c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.720+0000 7fe2adc00640 1 -- 172.21.15.103:0/2969919042 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4004d20 con 0x5588715cb1d0 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.720+0000 7fe2adc00640 1 -- 172.21.15.103:0/2969919042 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b4004050 con 0x5588715cb1d0 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.720+0000 7fe2adc00640 1 -- 172.21.15.103:0/2969919042 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4005220 con 0x5588715cb1d0 2024-09-11T00:32:23.748 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2969919042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 msgr2=0x5588715cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715cb5b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fd0 tx=0x7fe2b40118d0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2969919042 shutdown_connections 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715d7a80 0x5588715d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715cbaf0 0x5588715d7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2969919042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715cb5b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2969919042 >> 172.21.15.103:0/2969919042 conn(0x5588715c9000 msgr2=0x5588715ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2969919042 shutdown_connections 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2969919042 wait complete. 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2b9600640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2a40b52a0 0x7fe2a40b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2b9600640 1 -- 172.21.15.103:0/1204450563 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe2b00e63a0 con 0x7fe2bc0776d0 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2a40b52a0 0x7fe2a40b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.749 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.721+0000 7fe2c61c3740 1 -- start start 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 1 Processor -- start 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2a40b52a0 0x7fe2a40b7760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8031440 tx=0x7fe2a8005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558871392bb0 con 0x5588715cb1d0 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x5588714cc6c0 con 0x5588715d7a80 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5588715da780 con 0x5588715cbaf0 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.722+0000 7fe2c61c3740 10 client.0 _ll_register_callbacks cb 0x5588714e7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40718/0 (socket says 172.21.15.103:40718) 2024-09-11T00:32:23.750 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715d4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59442/0 (socket says 172.21.15.103:59442) 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2c0c00640 1 -- 172.21.15.103:0/1146518470 learned_addr learned my addr 172.21.15.103:0/1146518470 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2c1600640 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 -- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 msgr2=0x5588715d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 -- 172.21.15.103:0/1146518470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 msgr2=0x5588715d49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 -- 172.21.15.103:0/1146518470 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887128fb00 con 0x5588715cbaf0 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715d4f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009230 tx=0x7fe2a8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb65eaa05288c14f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.723+0000 7fe2a2000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2a2000640 1 -- 172.21.15.103:0/1146518470 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8035040 con 0x5588715cbaf0 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2b4004050 con 0x5588715cbaf0 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2a2000640 1 -- 172.21.15.103:0/1146518470 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe2a8005220 con 0x5588715cbaf0 2024-09-11T00:32:23.751 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2a2000640 1 -- 172.21.15.103:0/1146518470 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe2a8057050 con 0x5588715cbaf0 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2c61c3740 10 client.26401 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558871306960 con 0x5588715cbaf0 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2a2000640 1 -- 172.21.15.103:0/1146518470 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b4004050 con 0x5588715cbaf0 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2a2000640 1 -- 172.21.15.103:0/1146518470 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fe2a8003860 con 0x5588715cbaf0 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2c61c3740 10 client.26401 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2c61c3740 10 client.26401 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.724+0000 7fe2c61c3740 20 client.26401 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.725+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5588715e75e0 0x5588715e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.725+0000 7fe2c61c3740 4 client.26401 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.725+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588715441e0 con 0x5588715e75e0 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.725+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5588715e75e0 0x5588715e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.725+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5588715e75e0 0x5588715e9a40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853922c53f513024 server_cookie=5100171075d575f4 in_seq=0 out_seq=0 2024-09-11T00:32:23.752 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.725+0000 7fe2a2000640 10 client.26401 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2a2000640 1 -- 172.21.15.103:0/1146518470 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5588715441e0 con 0x5588715e75e0 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2a2000640 10 client.26401 handle_command_reply: tid=0 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 client.26401 shutdown 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 msgr2=0x5588715d4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715d4f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009230 tx=0x7fe2a8009260 comp rx=0 tx=0).stop 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 shutdown_connections 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x5588715e75e0 0x5588715e9a40 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715d4f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/1146518470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2a2000640 0 client.26401 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:23.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.726+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 >> 172.21.15.103:0/1146518470 conn(0x5588715c9000 msgr2=0x5588715ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.727+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 shutdown_connections 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.727+0000 7fe2c61c3740 1 -- 172.21.15.103:0/1146518470 wait complete. 2024-09-11T00:32:23.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.727+0000 7fe2c61c3740 20 client.26401 trim_cache size 0 max 16384 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 Processor -- start 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 -- start start 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b8720 con 0x5588715cb1d0 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b5ff0 con 0x5588715d7a80 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b3e40 con 0x5588715cbaf0 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.745+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:40732/0 (socket says 172.21.15.103:40732) 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2c1600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:53720/0 (socket says 172.21.15.103:53720) 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2c0c00640 1 -- 172.21.15.103:0/2992034435 learned_addr learned my addr 172.21.15.103:0/2992034435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2bbe00640 1 -- 172.21.15.103:0/2992034435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 msgr2=0x5588715d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/2992034435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2bbe00640 1 -- 172.21.15.103:0/2992034435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 msgr2=0x5588715d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/2992034435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2bbe00640 1 -- 172.21.15.103:0/2992034435 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588715c87d0 con 0x5588715cbaf0 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/2992034435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715cf240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8003cd0 tx=0x7fe2a8003ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1852b9ecdd0e601c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.746+0000 7fe2ad200640 1 -- 172.21.15.103:0/2992034435 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8032be0 con 0x5588715cbaf0 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2ad200640 1 -- 172.21.15.103:0/2992034435 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a805f030 con 0x5588715cbaf0 2024-09-11T00:32:23.759 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2ad200640 1 -- 172.21.15.103:0/2992034435 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a803dde0 con 0x5588715cbaf0 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2992034435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 msgr2=0x5588715cf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2992034435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715cf240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8003cd0 tx=0x7fe2a8003ec0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2992034435 shutdown_connections 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2992034435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2992034435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cbaf0 0x5588715cf240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2992034435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cb1d0 0x5588715d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2992034435 >> 172.21.15.103:0/2992034435 conn(0x5588715c9000 msgr2=0x5588715c80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2992034435 shutdown_connections 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.747+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2992034435 wait complete. 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.748+0000 7fe2c61c3740 1 -- start start 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.748+0000 7fe2c61c3740 1 Processor -- start 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cb1d0 0x5588715e0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.760 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cbaf0 0x5588715dc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c61c3740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b44b0 con 0x5588715cbaf0 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b5920 con 0x5588715d7a80 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c61c3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2bc1b4b80 con 0x5588715cb1d0 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c61c3740 10 client.0 _ll_register_callbacks cb 0x7fe2a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cb1d0 0x5588715e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cbaf0 0x5588715dc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cb1d0 0x5588715e0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:59458/0 (socket says 172.21.15.103:59458) 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 -- 172.21.15.103:0/2120794619 learned_addr learned my addr 172.21.15.103:0/2120794619 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c1600640 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 -- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 msgr2=0x5588715dcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715dcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.761 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 -- 172.21.15.103:0/2120794619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cbaf0 msgr2=0x5588715dc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cbaf0 0x5588715dc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.749+0000 7fe2c0c00640 1 -- 172.21.15.103:0/2120794619 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588715dd3d0 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2c0c00640 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cb1d0 0x5588715e0180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00ac700 tx=0x7fe2b000ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1527e90109c534a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2a2000640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2a2000640 1 -- 172.21.15.103:0/2120794619 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00ac540 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2a2000640 1 -- 172.21.15.103:0/2120794619 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe2b00a89e0 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a805f030 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2a2000640 1 -- 172.21.15.103:0/2120794619 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe2b00abee0 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2c61c3740 10 client.26407 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5588716555e0 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2a2000640 1 -- 172.21.15.103:0/2120794619 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a805f030 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2a2000640 1 -- 172.21.15.103:0/2120794619 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fe2b00a80c0 con 0x5588715cb1d0 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2c61c3740 10 client.26407 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:23.762 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.750+0000 7fe2c61c3740 10 client.26407 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.751+0000 7fe2c61c3740 20 client.26407 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.751+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558871655d10 0x558871658130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.751+0000 7fe2c61c3740 4 client.26407 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "19d99768", "release": true, "await": true}] 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.751+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "set_id": "19d99768", "release": true, "await": true}) -- 0x55887153fc90 con 0x558871655d10 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.751+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558871655d10 0x558871658130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.751+0000 7fe2bbe00640 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558871655d10 0x558871658130 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5946df1f12d6566 server_cookie=2401c97e52a060a7 in_seq=0 out_seq=0 2024-09-11T00:32:23.763 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.751+0000 7fe2a2000640 10 client.26407 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.879+0000 7fe2a2000640 1 -- 172.21.15.103:0/2120794619 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55887153fc90 con 0x558871655d10 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.879+0000 7fe2a2000640 10 client.26407 handle_command_reply: tid=0 2024-09-11T00:32:23.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 client.26407 shutdown 2024-09-11T00:32:23.882 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:23 smithi103 ceph-mon[30578]: pgmap v680: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-11T00:32:23.883 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cb1d0 msgr2=0x5588715e0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cb1d0 0x5588715e0180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00ac700 tx=0x7fe2b000ce60 comp rx=0 tx=0).stop 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 shutdown_connections 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x5588715d7a80 0x5588715dcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5588715cbaf0 0x5588715dc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558871655d10 0x558871658130 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 --2- 172.21.15.103:0/2120794619 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5588715cb1d0 0x5588715e0180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 >> 172.21.15.103:0/2120794619 conn(0x5588715c9000 msgr2=0x5588715e29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2a2000640 0 client.26407 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 shutdown_connections 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.880+0000 7fe2c61c3740 1 -- 172.21.15.103:0/2120794619 wait complete. 2024-09-11T00:32:23.884 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.881+0000 7fe2c61c3740 20 client.26407 trim_cache size 0 max 16384 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 44, 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 28, 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": { 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "19d99768": { 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "version": 28, 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "age_ref": 0.0, 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:32:23.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "timeout": 90.0, 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "expiration": 148.4, 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "members": { 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "file:/": { 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "excluded": false, 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "state": { 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RELEASED", 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: "age": 0.0 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:32:23.886 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2a40b52a0 msgr2=0x7fe2a40b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2a40b52a0 0x7fe2a40b7760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8031440 tx=0x7fe2a8005e50 comp rx=0 tx=0).stop 2024-09-11T00:32:23.887 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 msgr2=0x7fe2bc19f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 0x7fe2bc19f6b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000e9f0 tx=0x7fe2b000edf0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 shutdown_connections 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2a40b52a0 0x7fe2a40b7760 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2bc113800 0x7fe2bc1a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe2bc077ff0 0x7fe2bc19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 --2- 172.21.15.103:0/1204450563 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2bc0776d0 0x7fe2bc19f6b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 >> 172.21.15.103:0/1204450563 conn(0x7fe2bc100840 msgr2=0x7fe2bc102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 shutdown_connections 2024-09-11T00:32:23.888 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:23.882+0000 7fe2c2000640 1 -- 172.21.15.103:0/1204450563 wait complete. 2024-09-11T00:32:23.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:23 smithi117 ceph-mon[35525]: pgmap v680: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-11T00:32:24.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:23 smithi183 ceph-mon[35934]: pgmap v680: 129 pgs: 129 active+clean; 91 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-09-11T00:32:25.092 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '19d99768', total seconds elapsed: 32.8 2024-09-11T00:32:25.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:25 smithi117 ceph-mon[35525]: pgmap v681: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-11T00:32:26.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:25 smithi183 ceph-mon[35934]: pgmap v681: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-11T00:32:26.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:25 smithi103 ceph-mon[30578]: pgmap v681: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-11T00:32:27.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:27 smithi117 ceph-mon[35525]: pgmap v682: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-09-11T00:32:28.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:27 smithi183 ceph-mon[35934]: pgmap v682: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-09-11T00:32:28.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:27 smithi103 ceph-mon[30578]: pgmap v682: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-09-11T00:32:29.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:29 smithi117 ceph-mon[35525]: pgmap v683: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 224 KiB/s wr, 78 op/s 2024-09-11T00:32:30.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:29 smithi183 ceph-mon[35934]: pgmap v683: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 224 KiB/s wr, 78 op/s 2024-09-11T00:32:30.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:29 smithi103 ceph-mon[30578]: pgmap v683: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 224 KiB/s wr, 78 op/s 2024-09-11T00:32:31.937 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:31 smithi117 ceph-mon[35525]: pgmap v684: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 417 KiB/s wr, 142 op/s 2024-09-11T00:32:32.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:31 smithi183 ceph-mon[35934]: pgmap v684: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 417 KiB/s wr, 142 op/s 2024-09-11T00:32:32.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:31 smithi103 ceph-mon[30578]: pgmap v684: 129 pgs: 129 active+clean; 88 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 417 KiB/s wr, 142 op/s 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 -- 172.21.15.103:0/898250574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a8105cf0 msgr2=0x7f81a8106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 --2- 172.21.15.103:0/898250574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a8105cf0 0x7f81a8106170 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8194009930 tx=0x7f819402a420 comp rx=0 tx=0).stop 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 -- 172.21.15.103:0/898250574 shutdown_connections 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 --2- 172.21.15.103:0/898250574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81a81066b0 0x7f81a810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 --2- 172.21.15.103:0/898250574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a8105cf0 0x7f81a8106170 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 --2- 172.21.15.103:0/898250574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a8104ba0 0x7f81a8104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 -- 172.21.15.103:0/898250574 >> 172.21.15.103:0/898250574 conn(0x7f81a8100460 msgr2=0x7f81a8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 -- 172.21.15.103:0/898250574 shutdown_connections 2024-09-11T00:32:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.537+0000 7f81ae200640 1 -- 172.21.15.103:0/898250574 wait complete. 2024-09-11T00:32:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.538+0000 7f81ae200640 1 Processor -- start 2024-09-11T00:32:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81ae200640 1 -- start start 2024-09-11T00:32:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81ae200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a8104ba0 0x7f81a819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81ae200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81a8105cf0 0x7f81a819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81ae200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a81066b0 0x7f81a819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81ae200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8110840 con 0x7f81a8105cf0 2024-09-11T00:32:32.542 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81ae200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81106c0 con 0x7f81a8104ba0 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81ae200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81109c0 con 0x7f81a81066b0 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81a8105cf0 0x7f81a819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81a7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a81066b0 0x7f81a819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.539+0000 7f81a7e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a81066b0 0x7f81a819c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51316/0 (socket says 172.21.15.103:51316) 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a7e00640 1 -- 172.21.15.103:0/181472280 learned_addr learned my addr 172.21.15.103:0/181472280 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a7400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a8104ba0 0x7f81a819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a6a00640 1 -- 172.21.15.103:0/181472280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a81066b0 msgr2=0x7f81a819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a6a00640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a81066b0 0x7f81a819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a6a00640 1 -- 172.21.15.103:0/181472280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a8104ba0 msgr2=0x7f81a819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a6a00640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a8104ba0 0x7f81a819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a6a00640 1 -- 172.21.15.103:0/181472280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a819cb30 con 0x7f81a8105cf0 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a7e00640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a81066b0 0x7f81a819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a7400640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a8104ba0 0x7f81a819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.540+0000 7f81a6a00640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81a8105cf0 0x7f81a819b640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f819402a940 tx=0x7f8194003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.541+0000 7f819fe00640 1 -- 172.21.15.103:0/181472280 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819403f070 con 0x7f81a8105cf0 2024-09-11T00:32:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.541+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a819cdc0 con 0x7f81a8105cf0 2024-09-11T00:32:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.541+0000 7f819fe00640 1 -- 172.21.15.103:0/181472280 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8194031c50 con 0x7f81a8105cf0 2024-09-11T00:32:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.541+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a81a34f0 con 0x7f81a8105cf0 2024-09-11T00:32:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.541+0000 7f819fe00640 1 -- 172.21.15.103:0/181472280 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194049060 con 0x7f81a8105cf0 2024-09-11T00:32:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 Processor -- start 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 -- start start 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8dd5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8dd84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8ddafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a6f8d466b0 con 0x55a6f8dd8b00 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6f8d35ae0 con 0x55a6f8dd3790 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.544+0000 7f81b11c5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6f8d398f0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8dd84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8dd84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51330/0 (socket says 172.21.15.103:51330) 2024-09-11T00:32:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 -- 172.21.15.103:0/2817962983 learned_addr learned my addr 172.21.15.103:0/2817962983 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7e00640 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8dd5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 -- 172.21.15.103:0/2817962983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 msgr2=0x55a6f8dd5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8dd5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 -- 172.21.15.103:0/2817962983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 msgr2=0x55a6f8ddafc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8ddafc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 -- 172.21.15.103:0/2817962983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f8d02780 con 0x55a6f8dd60c0 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f81a7400640 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8dd84b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8198010480 tx=0x7f8198010880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7773abe6386866e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.545+0000 7f8193e00640 1 -- 172.21.15.103:0/2817962983 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198011700 con 0x55a6f8dd60c0 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f8193e00640 1 -- 172.21.15.103:0/2817962983 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8198011d50 con 0x55a6f8dd60c0 2024-09-11T00:32:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f8193e00640 1 -- 172.21.15.103:0/2817962983 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819801a380 con 0x55a6f8dd60c0 2024-09-11T00:32:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f819fe00640 1 -- 172.21.15.103:0/181472280 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f819403b4f0 con 0x7f81a8105cf0 2024-09-11T00:32:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f81b11c5740 1 -- 172.21.15.103:0/2817962983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 msgr2=0x55a6f8dd84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8dd84b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8198010480 tx=0x7f8198010880 comp rx=0 tx=0).stop 2024-09-11T00:32:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f81b11c5740 1 -- 172.21.15.103:0/2817962983 shutdown_connections 2024-09-11T00:32:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8ddafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8dd84b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2817962983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8dd5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.546+0000 7f81b11c5740 1 -- 172.21.15.103:0/2817962983 >> 172.21.15.103:0/2817962983 conn(0x55a6f8dce8e0 msgr2=0x55a6f8dcf670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.547+0000 7f81b11c5740 1 -- 172.21.15.103:0/2817962983 shutdown_connections 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.547+0000 7f81b11c5740 1 -- 172.21.15.103:0/2817962983 wait complete. 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.547+0000 7f81b11c5740 1 -- start start 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.547+0000 7f81b11c5740 1 Processor -- start 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8ddce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8ddd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8dde100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81b11c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a6f8c6dab0 con 0x55a6f8dd8b00 2024-09-11T00:32:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81b11c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x55a6f8cdc570 con 0x55a6f8dd3790 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81b11c5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a6f8ddbae0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81b11c5740 10 client.0 _ll_register_callbacks cb 0x55a6f8ceb100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f819fe00640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f818c0b51f0 0x7f818c0b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81a7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8ddd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81a7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8ddd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51338/0 (socket says 172.21.15.103:51338) 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81a7400640 1 -- 172.21.15.103:0/2835811609 learned_addr learned my addr 172.21.15.103:0/2835811609 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.548+0000 7f81a6a00640 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8dde100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f819fe00640 1 -- 172.21.15.103:0/181472280 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8194106ca0 con 0x7f81a8105cf0 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7e00640 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8ddce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7e00640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f818c0b51f0 0x7f818c0b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7400640 1 -- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 msgr2=0x55a6f8ddce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7400640 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8ddce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7400640 1 -- 172.21.15.103:0/2835811609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 msgr2=0x55a6f8dde100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7400640 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8dde100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7400640 1 -- 172.21.15.103:0/2835811609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f8a97790 con 0x55a6f8dd60c0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f81a7400640 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8ddd530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8198010450 tx=0x7f819801c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345437f238f9fc93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.549+0000 7f8190c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f8190c00640 1 -- 172.21.15.103:0/2835811609 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198011910 con 0x55a6f8dd60c0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81a7e00640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f818c0b51f0 0x7f818c0b76b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009aa0 tx=0x7f81a0012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8198011d50 con 0x55a6f8dd60c0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f8190c00640 1 -- 172.21.15.103:0/2835811609 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f8198011ab0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f8190c00640 1 -- 172.21.15.103:0/2835811609 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8198033050 con 0x55a6f8dd60c0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81b11c5740 10 client.26419 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a6f8b10620 con 0x55a6f8dd60c0 2024-09-11T00:32:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f8190c00640 1 -- 172.21.15.103:0/2835811609 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8198011d50 con 0x55a6f8dd60c0 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f8190c00640 1 -- 172.21.15.103:0/2835811609 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f819802e3b0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81b11c5740 10 client.26419 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81b11c5740 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81b11c5740 20 client.26419 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.550+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8de8500 0x55a6f8dea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.551+0000 7f81b11c5740 4 client.26419 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.551+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f8d48600 con 0x55a6f8de8500 2024-09-11T00:32:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.551+0000 7f81a6a00640 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8de8500 0x55a6f8dea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.551+0000 7f81a6a00640 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8de8500 0x55a6f8dea8f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64ec340a0ae9d8a8 server_cookie=fde322ec1e8af46d in_seq=0 out_seq=0 2024-09-11T00:32:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.551+0000 7f8190c00640 10 client.26419 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.552+0000 7f8190c00640 1 -- 172.21.15.103:0/2835811609 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6f8d48600 con 0x55a6f8de8500 2024-09-11T00:32:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.552+0000 7f8190c00640 10 client.26419 handle_command_reply: tid=0 2024-09-11T00:32:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.552+0000 7f81b11c5740 1 client.26419 shutdown 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 msgr2=0x55a6f8ddd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8ddd530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8198010450 tx=0x7f819801c730 comp rx=0 tx=0).stop 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 shutdown_connections 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8dde100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8de8500 0x55a6f8dea8f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd60c0 0x55a6f8ddd530 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 --2- 172.21.15.103:0/2835811609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd3790 0x55a6f8ddce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f8190c00640 0 client.26419 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 >> 172.21.15.103:0/2835811609 conn(0x55a6f8dce8e0 msgr2=0x55a6f8dcda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 shutdown_connections 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 1 -- 172.21.15.103:0/2835811609 wait complete. 2024-09-11T00:32:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.553+0000 7f81b11c5740 20 client.26419 trim_cache size 0 max 16384 2024-09-11T00:32:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 Processor -- start 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 -- start start 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8def140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd60c0 0x55a6f8def680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd8b00 0x55a6f8defbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81b01b0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ada80 con 0x55a6f8dd8b00 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.572+0000 7f81b11c5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ab8d0 con 0x55a6f8dd3790 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd60c0 0x55a6f8def680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8def140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a7400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8def140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:51340/0 (socket says 172.21.15.103:51340) 2024-09-11T00:32:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a7400640 1 -- 172.21.15.103:0/362453916 learned_addr learned my addr 172.21.15.103:0/362453916 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a7e00640 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd8b00 0x55a6f8defbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a6a00640 1 -- 172.21.15.103:0/362453916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 msgr2=0x55a6f8def140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a6a00640 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8def140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a6a00640 1 -- 172.21.15.103:0/362453916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd8b00 msgr2=0x55a6f8defbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a6a00640 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd8b00 0x55a6f8defbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a6a00640 1 -- 172.21.15.103:0/362453916 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f8ddc990 con 0x55a6f8dd60c0 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a7400640 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8def140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.573+0000 7f81a6a00640 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd60c0 0x55a6f8def680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f81941043b0 tx=0x7f8194005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693f61b5db5414d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.574+0000 7f8193400640 1 -- 172.21.15.103:0/362453916 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81940cd120 con 0x55a6f8dd60c0 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.574+0000 7f8193400640 1 -- 172.21.15.103:0/362453916 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81940d1b50 con 0x55a6f8dd60c0 2024-09-11T00:32:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.574+0000 7f8193400640 1 -- 172.21.15.103:0/362453916 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81940cd5e0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.574+0000 7f81b11c5740 1 -- 172.21.15.103:0/362453916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd60c0 msgr2=0x55a6f8def680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.574+0000 7f81b11c5740 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd60c0 0x55a6f8def680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f81941043b0 tx=0x7f8194005fd0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 -- 172.21.15.103:0/362453916 shutdown_connections 2024-09-11T00:32:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd8b00 0x55a6f8defbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd60c0 0x55a6f8def680 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 --2- 172.21.15.103:0/362453916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8def140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 -- 172.21.15.103:0/362453916 >> 172.21.15.103:0/362453916 conn(0x55a6f8dce8e0 msgr2=0x55a6f8dd2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 -- 172.21.15.103:0/362453916 shutdown_connections 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 -- 172.21.15.103:0/362453916 wait complete. 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 -- start start 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.575+0000 7f81b11c5740 1 Processor -- start 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8de4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd60c0 0x55a6f8de4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81b11c5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8de2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81b11c5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81abf40 con 0x55a6f8dd8b00 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81b11c5740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ad3b0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81b11c5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f81a81ac610 con 0x55a6f8dd3790 2024-09-11T00:32:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81b11c5740 10 client.0 _ll_register_callbacks cb 0x7f819800bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81a7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8de2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81a7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8de2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:51536/0 (socket says 172.21.15.103:51536) 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81a7e00640 1 -- 172.21.15.103:0/433802220 learned_addr learned my addr 172.21.15.103:0/433802220 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.576+0000 7f81a6a00640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd60c0 0x55a6f8de4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a7400640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8de4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a6a00640 1 -- 172.21.15.103:0/433802220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 msgr2=0x55a6f8de4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a6a00640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8de4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a6a00640 1 -- 172.21.15.103:0/433802220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 msgr2=0x55a6f8de2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a6a00640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8de2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a6a00640 1 -- 172.21.15.103:0/433802220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f8deaf20 con 0x55a6f8dd60c0 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a7e00640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8de2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81a6a00640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd60c0 0x55a6f8de4b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f81940d67c0 tx=0x7f819402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5a91010661a9255 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f8190c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f8190c00640 1 -- 172.21.15.103:0/433802220 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81940f00b0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f8190c00640 1 -- 172.21.15.103:0/433802220 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f81940d6170 con 0x55a6f8dd60c0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.577+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81940d1b50 con 0x55a6f8dd60c0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f8190c00640 1 -- 172.21.15.103:0/433802220 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f81940cd120 con 0x55a6f8dd60c0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 10 client.26405 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a6f8e55dd0 con 0x55a6f8dd60c0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f8190c00640 1 -- 172.21.15.103:0/433802220 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81940d1b50 con 0x55a6f8dd60c0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f8190c00640 1 -- 172.21.15.103:0/433802220 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7f81940c8a70 con 0x55a6f8dd60c0 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 10 client.26405 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 10 client.26405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 20 client.26405 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8e56700 0x55a6f8e58b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 4 client.26405 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:32:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a6f8d440b0 con 0x55a6f8e56700 2024-09-11T00:32:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.578+0000 7f81a7400640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8e56700 0x55a6f8e58b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.579+0000 7f81a7400640 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8e56700 0x55a6f8e58b20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4feda013b3c8fc8 server_cookie=cbd7e2e5e586aa11 in_seq=0 out_seq=0 2024-09-11T00:32:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.579+0000 7f8190c00640 10 client.26405 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f8190c00640 1 -- 172.21.15.103:0/433802220 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a6f8d440b0 con 0x55a6f8e56700 2024-09-11T00:32:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f8190c00640 10 client.26405 handle_command_reply: tid=0 2024-09-11T00:32:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 client.26405 shutdown 2024-09-11T00:32:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd60c0 msgr2=0x55a6f8de4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd60c0 0x55a6f8de4b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f81940d67c0 tx=0x7f819402a420 comp rx=0 tx=0).stop 2024-09-11T00:32:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 shutdown_connections 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a6f8dd8b00 0x55a6f8de2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55a6f8dd60c0 0x55a6f8de4b00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x55a6f8e56700 0x55a6f8e58b20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 --2- 172.21.15.103:0/433802220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a6f8dd3790 0x55a6f8de4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f8190c00640 0 client.26405 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 >> 172.21.15.103:0/433802220 conn(0x55a6f8dce8e0 msgr2=0x55a6f8dcd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 shutdown_connections 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 1 -- 172.21.15.103:0/433802220 wait complete. 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.590+0000 7f81b11c5740 20 client.26405 trim_cache size 0 max 16384 2024-09-11T00:32:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f818c0b51f0 msgr2=0x7f818c0b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f818c0b51f0 0x7f818c0b76b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009aa0 tx=0x7f81a0012040 comp rx=0 tx=0).stop 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81a8105cf0 msgr2=0x7f81a819b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81a8105cf0 0x7f81a819b640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f819402a940 tx=0x7f8194003c10 comp rx=0 tx=0).stop 2024-09-11T00:32:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 shutdown_connections 2024-09-11T00:32:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f818c0b51f0 0x7f818c0b76b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f81a81066b0 0x7f81a819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81a8105cf0 0x7f81a819b640 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 --2- 172.21.15.103:0/181472280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81a8104ba0 0x7f81a819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 >> 172.21.15.103:0/181472280 conn(0x7f81a8100460 msgr2=0x7f81a8107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 shutdown_connections 2024-09-11T00:32:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:32.591+0000 7f81ae200640 1 -- 172.21.15.103:0/181472280 wait complete. 2024-09-11T00:32:34.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:33 smithi183 ceph-mon[35934]: pgmap v685: 129 pgs: 129 active+clean; 86 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 623 KiB/s wr, 219 op/s 2024-09-11T00:32:34.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:33 smithi103 ceph-mon[30578]: pgmap v685: 129 pgs: 129 active+clean; 86 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 623 KiB/s wr, 219 op/s 2024-09-11T00:32:34.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:33 smithi117 ceph-mon[35525]: pgmap v685: 129 pgs: 129 active+clean; 86 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 623 KiB/s wr, 219 op/s 2024-09-11T00:32:36.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:35 smithi183 ceph-mon[35934]: pgmap v686: 129 pgs: 129 active+clean; 81 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 858 KiB/s wr, 312 op/s 2024-09-11T00:32:36.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:35 smithi103 ceph-mon[30578]: pgmap v686: 129 pgs: 129 active+clean; 81 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 858 KiB/s wr, 312 op/s 2024-09-11T00:32:36.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:35 smithi117 ceph-mon[35525]: pgmap v686: 129 pgs: 129 active+clean; 81 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 858 KiB/s wr, 312 op/s 2024-09-11T00:32:38.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:37 smithi183 ceph-mon[35934]: pgmap v687: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 977 KiB/s wr, 354 op/s 2024-09-11T00:32:38.169 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:37 smithi103 ceph-mon[30578]: pgmap v687: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 977 KiB/s wr, 354 op/s 2024-09-11T00:32:38.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:37 smithi117 ceph-mon[35525]: pgmap v687: 129 pgs: 129 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 977 KiB/s wr, 354 op/s 2024-09-11T00:32:40.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:39 smithi183 ceph-mon[35934]: pgmap v688: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 408 op/s 2024-09-11T00:32:40.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:39 smithi103 ceph-mon[30578]: pgmap v688: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 408 op/s 2024-09-11T00:32:40.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:39 smithi117 ceph-mon[35525]: pgmap v688: 129 pgs: 129 active+clean; 72 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 408 op/s 2024-09-11T00:32:41.476 INFO:tasks.workunit.client.0.smithi103.stdout:All 50000 operations completed A-OK! 2024-09-11T00:32:41.509 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T00:32:41.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T00:32:41.919 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:41 smithi103 ceph-mon[30578]: pgmap v689: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 419 op/s 2024-09-11T00:32:42.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:41 smithi183 ceph-mon[35934]: pgmap v689: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 419 op/s 2024-09-11T00:32:42.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:41 smithi117 ceph-mon[35525]: pgmap v689: 129 pgs: 129 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 1.1 MiB/s wr, 419 op/s 2024-09-11T00:32:44.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:43 smithi183 ceph-mon[35934]: pgmap v690: 129 pgs: 129 active+clean; 81 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-09-11T00:32:44.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:43 smithi103 ceph-mon[30578]: pgmap v690: 129 pgs: 129 active+clean; 81 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-09-11T00:32:44.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:43 smithi117 ceph-mon[35525]: pgmap v690: 129 pgs: 129 active+clean; 81 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.1 MiB/s wr, 427 op/s 2024-09-11T00:32:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 -- 172.21.15.103:0/1504165643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf01038b0 msgr2=0x7fdcf0105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 --2- 172.21.15.103:0/1504165643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf01038b0 0x7fdcf0105d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009900 tx=0x7fdce002a3f0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 -- 172.21.15.103:0/1504165643 shutdown_connections 2024-09-11T00:32:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 --2- 172.21.15.103:0/1504165643 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcf0106460 0x7fdcf0106910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 --2- 172.21.15.103:0/1504165643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf01038b0 0x7fdcf0105d20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 --2- 172.21.15.103:0/1504165643 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf006b470 0x7fdcf0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 -- 172.21.15.103:0/1504165643 >> 172.21.15.103:0/1504165643 conn(0x7fdcf00fd140 msgr2=0x7fdcf00ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 -- 172.21.15.103:0/1504165643 shutdown_connections 2024-09-11T00:32:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.766+0000 7fdcf6000640 1 -- 172.21.15.103:0/1504165643 wait complete. 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.768+0000 7fdcf6000640 1 Processor -- start 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.768+0000 7fdcf6000640 1 -- start start 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.768+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf006b470 0x7fdcf010c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.768+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcf01038b0 0x7fdcf010ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.768+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf0106460 0x7fdcf010da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcf6000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114bf0 con 0x7fdcf01038b0 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcf6000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114a70 con 0x7fdcf006b470 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcf6000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114d70 con 0x7fdcf0106460 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf0106460 0x7fdcf010da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.771 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf0106460 0x7fdcf010da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:42828/0 (socket says 172.21.15.103:42828) 2024-09-11T00:32:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 -- 172.21.15.103:0/2151898419 learned_addr learned my addr 172.21.15.103:0/2151898419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcef400640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf006b470 0x7fdcf010c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 -- 172.21.15.103:0/2151898419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf006b470 msgr2=0x7fdcf010c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf006b470 0x7fdcf010c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 -- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcf01038b0 msgr2=0x7fdcf010ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcf01038b0 0x7fdcf010ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.769+0000 7fdcefe00640 1 -- 172.21.15.103:0/2151898419 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf010e320 con 0x7fdcf0106460 2024-09-11T00:32:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.770+0000 7fdcefe00640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf0106460 0x7fdcf010da60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdce800eaa0 tx=0x7fdce800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.771+0000 7fdce7e00640 1 -- 172.21.15.103:0/2151898419 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce800cdb0 con 0x7fdcf0106460 2024-09-11T00:32:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.771+0000 7fdce7e00640 1 -- 172.21.15.103:0/2151898419 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdce8005020 con 0x7fdcf0106460 2024-09-11T00:32:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.771+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf010e610 con 0x7fdcf0106460 2024-09-11T00:32:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.772+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf010ecb0 con 0x7fdcf0106460 2024-09-11T00:32:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.772+0000 7fdce7e00640 1 -- 172.21.15.103:0/2151898419 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80116d0 con 0x7fdcf0106460 2024-09-11T00:32:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.777+0000 7fdcf906c740 1 Processor -- start 2024-09-11T00:32:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.779+0000 7fdce7e00640 1 -- 172.21.15.103:0/2151898419 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fdce800f040 con 0x7fdcf0106460 2024-09-11T00:32:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.780+0000 7fdcf906c740 1 -- start start 2024-09-11T00:32:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.780+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 0x558f3e648b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e6490c0 0x558f3e64b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e64bb00 0x558f3e64dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcf906c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558f3e5b96b0 con 0x558f3e646790 2024-09-11T00:32:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcf906c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f3e5a8ae0 con 0x558f3e6490c0 2024-09-11T00:32:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcf906c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f3e5ac8f0 con 0x558f3e64bb00 2024-09-11T00:32:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdce7e00640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdcd40b5430 0x7fdcd40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e6490c0 0x558f3e64b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdce7e00640 1 -- 172.21.15.103:0/2151898419 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdce80e5cc0 con 0x7fdcf0106460 2024-09-11T00:32:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e6490c0 0x558f3e64b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:47370/0 (socket says 172.21.15.103:47370) 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.781+0000 7fdcef400640 1 -- 172.21.15.103:0/1046353144 learned_addr learned my addr 172.21.15.103:0/1046353144 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.782+0000 7fdcefe00640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdcd40b5430 0x7fdcd40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.782+0000 7fdceea00640 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e64bb00 0x558f3e64dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.782+0000 7fdcef400640 1 -- 172.21.15.103:0/1046353144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e64bb00 msgr2=0x558f3e64dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.782+0000 7fdcef400640 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e64bb00 0x558f3e64dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.782+0000 7fdcef400640 1 -- 172.21.15.103:0/1046353144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 msgr2=0x558f3e648b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.782+0000 7fdcef400640 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 0x558f3e648b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.782+0000 7fdcef400640 1 -- 172.21.15.103:0/1046353144 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3e575780 con 0x558f3e6490c0 2024-09-11T00:32:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.783+0000 7fdcef400640 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e6490c0 0x558f3e64b4b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009cf0 tx=0x7fdcdc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aff59b638ba18aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.784+0000 7fdcdbe00640 1 -- 172.21.15.103:0/1046353144 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004230 con 0x558f3e6490c0 2024-09-11T00:32:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.784+0000 7fdcdbe00640 1 -- 172.21.15.103:0/1046353144 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc0043d0 con 0x558f3e6490c0 2024-09-11T00:32:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.784+0000 7fdcdbe00640 1 -- 172.21.15.103:0/1046353144 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc019880 con 0x558f3e6490c0 2024-09-11T00:32:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcf906c740 1 -- 172.21.15.103:0/1046353144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e6490c0 msgr2=0x558f3e64b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcefe00640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdcd40b5430 0x7fdcd40b78f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdce80ac080 tx=0x7fdce800cf50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e6490c0 0x558f3e64b4b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009cf0 tx=0x7fdcdc011810 comp rx=0 tx=0).stop 2024-09-11T00:32:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcf906c740 1 -- 172.21.15.103:0/1046353144 shutdown_connections 2024-09-11T00:32:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e64bb00 0x558f3e64dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e6490c0 0x558f3e64b4b0 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009cf0 tx=0x7fdcdc011810 comp rx=0 tx=0).stop 2024-09-11T00:32:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1046353144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 0x558f3e648b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.785+0000 7fdcf906c740 1 -- 172.21.15.103:0/1046353144 >> 172.21.15.103:0/1046353144 conn(0x558f3e6418e0 msgr2=0x558f3e642670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:45.788 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.786+0000 7fdcf906c740 1 -- 172.21.15.103:0/1046353144 shutdown_connections 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.787+0000 7fdcf906c740 1 -- 172.21.15.103:0/1046353144 wait complete. 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.787+0000 7fdcf906c740 1 -- start start 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.788+0000 7fdcf906c740 1 Processor -- start 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.788+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e650100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.788+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e6507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.788+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e651350 0x558f3e6517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.788+0000 7fdcf906c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558f3e4e0ab0 con 0x558f3e651350 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.788+0000 7fdcf906c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x558f3e54f570 con 0x558f3e64bb00 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.788+0000 7fdcf906c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558f3e64eae0 con 0x558f3e646790 2024-09-11T00:32:45.791 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.789+0000 7fdcf906c740 10 client.0 _ll_register_callbacks cb 0x558f3e55e100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.789+0000 7fdceea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e650100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.789+0000 7fdceea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e651350 0x558f3e6517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.789+0000 7fdceea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e651350 0x558f3e6517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:44820/0 (socket says 172.21.15.103:44820) 2024-09-11T00:32:45.792 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.789+0000 7fdceea00640 1 -- 172.21.15.103:0/3702002802 learned_addr learned my addr 172.21.15.103:0/3702002802 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.790+0000 7fdceea00640 1 -- 172.21.15.103:0/3702002802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 msgr2=0x558f3e6507a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:32:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.790+0000 7fdceea00640 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e6507a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.790+0000 7fdceea00640 1 -- 172.21.15.103:0/3702002802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e651350 msgr2=0x558f3e6517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.790+0000 7fdceea00640 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e651350 0x558f3e6517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.790+0000 7fdceea00640 1 -- 172.21.15.103:0/3702002802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3e30a790 con 0x558f3e646790 2024-09-11T00:32:45.793 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.791+0000 7fdceea00640 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e650100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009230 tx=0x7fdce0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65aeccf5d76a9002 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.791+0000 7fdcd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-11T00:32:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.791+0000 7fdcd8c00640 1 -- 172.21.15.103:0/3702002802 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0035040 con 0x558f3e646790 2024-09-11T00:32:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.792+0000 7fdcd8c00640 1 -- 172.21.15.103:0/3702002802 <== mon.1 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdce0004230 con 0x558f3e646790 2024-09-11T00:32:45.794 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.792+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcdc0043d0 con 0x558f3e646790 2024-09-11T00:32:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.792+0000 7fdcd8c00640 1 -- 172.21.15.103:0/3702002802 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdce004b050 con 0x558f3e646790 2024-09-11T00:32:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.792+0000 7fdcf906c740 10 client.26437 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:45.795 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.792+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558f3e383620 con 0x558f3e646790 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.793+0000 7fdcd8c00640 1 -- 172.21.15.103:0/3702002802 <== mon.1 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc0043d0 con 0x558f3e646790 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.794+0000 7fdcd8c00640 1 -- 172.21.15.103:0/3702002802 <== mon.1 v2:172.21.15.183:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fdce003d4f0 con 0x558f3e646790 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.794+0000 7fdcf906c740 10 client.26437 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.794+0000 7fdcf906c740 10 client.26437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.794+0000 7fdcf906c740 20 client.26437 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.795+0000 7fdcf906c740 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e65be50 0x558f3e65e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.795+0000 7fdcf906c740 4 client.26437 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:45.797 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.795+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f3e5bb600 con 0x558f3e65be50 2024-09-11T00:32:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.795+0000 7fdcef400640 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e65be50 0x558f3e65e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.796+0000 7fdcef400640 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e65be50 0x558f3e65e2d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32ea5c4524f6b278 server_cookie=678a94e1d8422d6 in_seq=0 out_seq=0 2024-09-11T00:32:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.796+0000 7fdcd8c00640 10 client.26437 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.797+0000 7fdcd8c00640 1 -- 172.21.15.103:0/3702002802 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558f3e5bb600 con 0x558f3e65be50 2024-09-11T00:32:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.797+0000 7fdcd8c00640 10 client.26437 handle_command_reply: tid=0 2024-09-11T00:32:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.797+0000 7fdcf906c740 1 client.26437 shutdown 2024-09-11T00:32:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 msgr2=0x558f3e650100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e650100 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009230 tx=0x7fdce0005850 comp rx=0 tx=0).stop 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 shutdown_connections 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e651350 0x558f3e6517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e6507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e65be50 0x558f3e65e2d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 --2- 172.21.15.103:0/3702002802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e650100 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 >> 172.21.15.103:0/3702002802 conn(0x558f3e6418e0 msgr2=0x558f3e640a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.798+0000 7fdcd8c00640 0 client.26437 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.799+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 shutdown_connections 2024-09-11T00:32:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.799+0000 7fdcf906c740 1 -- 172.21.15.103:0/3702002802 wait complete. 2024-09-11T00:32:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.799+0000 7fdcf906c740 20 client.26437 trim_cache size 0 max 16384 2024-09-11T00:32:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.834+0000 7fdcf906c740 1 Processor -- start 2024-09-11T00:32:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcf906c740 1 -- start start 2024-09-11T00:32:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 0x558f3e642960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e64c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e651350 0x558f3e6437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcf906c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd0012490 con 0x558f3e646790 2024-09-11T00:32:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcf906c740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd000fd60 con 0x558f3e64bb00 2024-09-11T00:32:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcf906c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd000dbb0 con 0x558f3e651350 2024-09-11T00:32:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e64c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcef400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e64c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:47384/0 (socket says 172.21.15.103:47384) 2024-09-11T00:32:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdcef400640 1 -- 172.21.15.103:0/4138809204 learned_addr learned my addr 172.21.15.103:0/4138809204 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.835+0000 7fdceea00640 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 0x558f3e642960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdceea00640 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e651350 0x558f3e6437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdcef400640 1 -- 172.21.15.103:0/4138809204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e651350 msgr2=0x558f3e6437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdcef400640 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e651350 0x558f3e6437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdcef400640 1 -- 172.21.15.103:0/4138809204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 msgr2=0x558f3e642960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdcef400640 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 0x558f3e642960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdcef400640 1 -- 172.21.15.103:0/4138809204 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f3e64fc00 con 0x558f3e64bb00 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdceea00640 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e651350 0x558f3e6437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.836+0000 7fdcef400640 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e64c5f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc001eb0 tx=0x7fdcdc009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2f494ae62cc114a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.837+0000 7fdcdb400640 1 -- 172.21.15.103:0/4138809204 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc02a070 con 0x558f3e64bb00 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.837+0000 7fdcdb400640 1 -- 172.21.15.103:0/4138809204 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc005bd0 con 0x558f3e64bb00 2024-09-11T00:32:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.837+0000 7fdcdb400640 1 -- 172.21.15.103:0/4138809204 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc02d690 con 0x558f3e64bb00 2024-09-11T00:32:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 -- 172.21.15.103:0/4138809204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 msgr2=0x558f3e64c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e64c5f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc001eb0 tx=0x7fdcdc009c80 comp rx=0 tx=0).stop 2024-09-11T00:32:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 -- 172.21.15.103:0/4138809204 shutdown_connections 2024-09-11T00:32:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e651350 0x558f3e6437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e64bb00 0x558f3e64c5f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 --2- 172.21.15.103:0/4138809204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e646790 0x558f3e642960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 -- 172.21.15.103:0/4138809204 >> 172.21.15.103:0/4138809204 conn(0x558f3e6418e0 msgr2=0x558f3e64bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.838+0000 7fdcf906c740 1 -- 172.21.15.103:0/4138809204 shutdown_connections 2024-09-11T00:32:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.839+0000 7fdcf906c740 1 -- 172.21.15.103:0/4138809204 wait complete. 2024-09-11T00:32:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.839+0000 7fdcf906c740 1 -- start start 2024-09-11T00:32:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.839+0000 7fdcf906c740 1 Processor -- start 2024-09-11T00:32:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e6571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e64bb00 0x558f3e657730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdcf906c740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e651350 0x558f3e657cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdcf906c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd000e220 con 0x558f3e64bb00 2024-09-11T00:32:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdceea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e6571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdceea00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e6571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:42850/0 (socket says 172.21.15.103:42850) 2024-09-11T00:32:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdceea00640 1 -- 172.21.15.103:0/1199492188 learned_addr learned my addr 172.21.15.103:0/1199492188 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd000f690 con 0x558f3e651350 2024-09-11T00:32:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd000e8f0 con 0x558f3e646790 2024-09-11T00:32:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.841+0000 7fdcf906c740 10 client.0 _ll_register_callbacks cb 0x7fdcdc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.842+0000 7fdceea00640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e651350 0x558f3e657cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.842+0000 7fdcef400640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e64bb00 0x558f3e657730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.842+0000 7fdceea00640 1 -- 172.21.15.103:0/1199492188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 msgr2=0x558f3e6571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.842+0000 7fdceea00640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e6571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.842+0000 7fdceea00640 1 -- 172.21.15.103:0/1199492188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e64bb00 msgr2=0x558f3e657730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.842+0000 7fdceea00640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e64bb00 0x558f3e657730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.843+0000 7fdceea00640 1 -- 172.21.15.103:0/1199492188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f3e6560c0 con 0x558f3e651350 2024-09-11T00:32:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.843+0000 7fdcef400640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e64bb00 0x558f3e657730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T00:32:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.843+0000 7fdceea00640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e651350 0x558f3e657cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdce003d940 tx=0x7fdce0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cbd0f9b8dabeb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.843+0000 7fdcd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:32:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.843+0000 7fdcd8c00640 1 -- 172.21.15.103:0/1199492188 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0035070 con 0x558f3e651350 2024-09-11T00:32:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.843+0000 7fdcd8c00640 1 -- 172.21.15.103:0/1199492188 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fdce0004040 con 0x558f3e651350 2024-09-11T00:32:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.844+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcdc005bd0 con 0x558f3e651350 2024-09-11T00:32:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.845+0000 7fdcd8c00640 1 -- 172.21.15.103:0/1199492188 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fdce003d690 con 0x558f3e651350 2024-09-11T00:32:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.845+0000 7fdcf906c740 10 client.26420 fetch_fsmap learned FSMap version 44 2024-09-11T00:32:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.845+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x558f3e6c9410 con 0x558f3e651350 2024-09-11T00:32:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.846+0000 7fdcd8c00640 1 -- 172.21.15.103:0/1199492188 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc005bd0 con 0x558f3e651350 2024-09-11T00:32:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.846+0000 7fdcd8c00640 1 -- 172.21.15.103:0/1199492188 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 44) ==== 8354+0+0 (secure 0 0 0) 0x7fdce0048050 con 0x558f3e651350 2024-09-11T00:32:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.847+0000 7fdcf906c740 10 client.26420 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T00:32:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.847+0000 7fdcf906c740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.847+0000 7fdcf906c740 20 client.26420 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.847+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e6c9e20 0x558f3e6cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.847+0000 7fdcf906c740 4 client.26420 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:32:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.847+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f3e5b70b0 con 0x558f3e6c9e20 2024-09-11T00:32:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.847+0000 7fdcef400640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e6c9e20 0x558f3e6cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.848+0000 7fdcd8c00640 10 client.26420 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.848+0000 7fdcef400640 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e6c9e20 0x558f3e6cc240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=497b0ddb35aa6446 server_cookie=8a0989c048416aa2 in_seq=0 out_seq=0 2024-09-11T00:32:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcd8c00640 1 -- 172.21.15.103:0/1199492188 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f3e5b70b0 con 0x558f3e6c9e20 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcd8c00640 10 client.26420 handle_command_reply: tid=0 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 client.26420 shutdown 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e651350 msgr2=0x558f3e657cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e651350 0x558f3e657cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdce003d940 tx=0x7fdce0005850 comp rx=0 tx=0).stop 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 shutdown_connections 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x558f3e651350 0x558f3e657cb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558f3e64bb00 0x558f3e657730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x558f3e6c9e20 0x558f3e6cc240 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 --2- 172.21.15.103:0/1199492188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558f3e646790 0x558f3e6571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 >> 172.21.15.103:0/1199492188 conn(0x558f3e6418e0 msgr2=0x558f3e640240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcd8c00640 0 client.26420 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 shutdown_connections 2024-09-11T00:32:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 1 -- 172.21.15.103:0/1199492188 wait complete. 2024-09-11T00:32:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.877+0000 7fdcf906c740 20 client.26420 trim_cache size 0 max 16384 2024-09-11T00:32:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:32:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:32:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:32:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:32:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdcd40b5430 msgr2=0x7fdcd40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdcd40b5430 0x7fdcd40b78f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdce80ac080 tx=0x7fdce800cf50 comp rx=0 tx=0).stop 2024-09-11T00:32:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf0106460 msgr2=0x7fdcf010da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf0106460 0x7fdcf010da60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdce800eaa0 tx=0x7fdce800ef60 comp rx=0 tx=0).stop 2024-09-11T00:32:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 shutdown_connections 2024-09-11T00:32:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fdcd40b5430 0x7fdcd40b78f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcf0106460 0x7fdcf010da60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcf01038b0 0x7fdcf010ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 --2- 172.21.15.103:0/2151898419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf006b470 0x7fdcf010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 >> 172.21.15.103:0/2151898419 conn(0x7fdcf00fd140 msgr2=0x7fdcf0101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.878+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 shutdown_connections 2024-09-11T00:32:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:45.879+0000 7fdcf6000640 1 -- 172.21.15.103:0/2151898419 wait complete. 2024-09-11T00:32:45.989 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:45 smithi117 ceph-mon[35525]: pgmap v691: 129 pgs: 129 active+clean; 81 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-09-11T00:32:46.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:45 smithi183 ceph-mon[35934]: pgmap v691: 129 pgs: 129 active+clean; 81 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-09-11T00:32:46.171 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:45 smithi103 ceph-mon[30578]: pgmap v691: 129 pgs: 129 active+clean; 81 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.1 MiB/s wr, 417 op/s 2024-09-11T00:32:47.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:47 smithi103 ceph-mon[30578]: pgmap v692: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 919 KiB/s wr, 344 op/s 2024-09-11T00:32:47.996 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:47 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:32:48.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:47 smithi183 ceph-mon[35934]: pgmap v692: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 919 KiB/s wr, 344 op/s 2024-09-11T00:32:48.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:47 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:32:48.035 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:47 smithi117 ceph-mon[35525]: pgmap v692: 129 pgs: 129 active+clean; 71 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 919 KiB/s wr, 344 op/s 2024-09-11T00:32:48.035 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:47 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-11T00:32:49.030 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:48 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:49.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:48 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:48 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:48 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.112 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:48 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:49.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:48 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:48 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.113 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:48 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:48 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:49.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:48 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:48 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:49.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:48 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:49 smithi183 ceph-mon[35934]: pgmap v693: 129 pgs: 129 active+clean; 68 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-09-11T00:32:50.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:49 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:49 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.031 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:49 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:49 smithi103 ceph-mon[30578]: pgmap v693: 129 pgs: 129 active+clean; 68 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-09-11T00:32:50.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.170 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:49 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:49 smithi117 ceph-mon[35525]: pgmap v693: 129 pgs: 129 active+clean; 68 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.1 MiB/s wr, 371 op/s 2024-09-11T00:32:50.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:49 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:49 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:50.187 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:49 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:51.512 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:51 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:51.512 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:51 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:51.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:51 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:51.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:51 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:51 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:51.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:51 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:52.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:52 smithi183 ceph-mon[35934]: pgmap v694: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 376 op/s 2024-09-11T00:32:52.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:52 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:32:52.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:52 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:32:52.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:52 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:52 smithi103 ceph-mon[30578]: pgmap v694: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 376 op/s 2024-09-11T00:32:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:52 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:32:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:52 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:32:52.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:52 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:52 smithi117 ceph-mon[35525]: pgmap v694: 129 pgs: 129 active+clean; 65 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 376 op/s 2024-09-11T00:32:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:52 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-11T00:32:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:52 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-11T00:32:52.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:52 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' 2024-09-11T00:32:54.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:54 smithi103 ceph-mon[30578]: pgmap v695: 129 pgs: 129 active+clean; 64 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1017 KiB/s wr, 405 op/s 2024-09-11T00:32:54.472 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:54 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:54.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:54 smithi183 ceph-mon[35934]: pgmap v695: 129 pgs: 129 active+clean; 64 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1017 KiB/s wr, 405 op/s 2024-09-11T00:32:54.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:54 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:54 smithi117 ceph-mon[35525]: pgmap v695: 129 pgs: 129 active+clean; 64 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1017 KiB/s wr, 405 op/s 2024-09-11T00:32:54.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:54 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:55.893 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-09-11T00:32:55.893 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T00:32:56.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:56 smithi183 ceph-mon[35934]: pgmap v696: 129 pgs: 129 active+clean; 69 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 459 op/s 2024-09-11T00:32:56.554 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:56 smithi103 ceph-mon[30578]: pgmap v696: 129 pgs: 129 active+clean; 69 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 459 op/s 2024-09-11T00:32:56.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:56 smithi117 ceph-mon[35525]: pgmap v696: 129 pgs: 129 active+clean; 69 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 459 op/s 2024-09-11T00:32:57.006 DEBUG:teuthology.parallel:result is None 2024-09-11T00:32:57.006 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T00:32:57.073 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T00:32:57.073 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T00:32:57.083 INFO:tasks.quiescer:joining Quiescers 2024-09-11T00:32:57.084 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T00:32:57.084 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:32:57.463 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:58.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 -- 172.21.15.103:0/645526057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f641093d0 msgr2=0x7f6f641097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 --2- 172.21.15.103:0/645526057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f641093d0 0x7f6f641097b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009ca0 tx=0x7f6f5402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 -- 172.21.15.103:0/645526057 shutdown_connections 2024-09-11T00:32:58.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 --2- 172.21.15.103:0/645526057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6410f760 0x7f6f64115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 --2- 172.21.15.103:0/645526057 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f6410ede0 0x7f6f6410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 --2- 172.21.15.103:0/645526057 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f641093d0 0x7f6f641097b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.070 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 -- 172.21.15.103:0/645526057 >> 172.21.15.103:0/645526057 conn(0x7f6f64078a40 msgr2=0x7f6f6407ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:58.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 -- 172.21.15.103:0/645526057 shutdown_connections 2024-09-11T00:32:58.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.067+0000 7f6f6b200640 1 -- 172.21.15.103:0/645526057 wait complete. 2024-09-11T00:32:58.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 Processor -- start 2024-09-11T00:32:58.071 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 -- start start 2024-09-11T00:32:58.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f641093d0 0x7f6f6419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6410ede0 0x7f6f6419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.072 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6410f760 0x7f6f641a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64119390 con 0x7f6f6410f760 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64119210 con 0x7f6f6410ede0 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.068+0000 7f6f6b200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6f64119510 con 0x7f6f641093d0 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6410f760 0x7f6f641a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6410f760 0x7f6f641a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41876/0 (socket says 172.21.15.103:41876) 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 -- 172.21.15.103:0/1516767435 learned_addr learned my addr 172.21.15.103:0/1516767435 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f63e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f641093d0 0x7f6f6419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f63400640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6410ede0 0x7f6f6419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 -- 172.21.15.103:0/1516767435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f641093d0 msgr2=0x7f6f6419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f641093d0 0x7f6f6419f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 -- 172.21.15.103:0/1516767435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6410ede0 msgr2=0x7f6f6419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6410ede0 0x7f6f6419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.069+0000 7f6f69200640 1 -- 172.21.15.103:0/1516767435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f641a5970 con 0x7f6f6410f760 2024-09-11T00:32:58.073 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.070+0000 7f6f69200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6410f760 0x7f6f641a51a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d950 tx=0x7f6f5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:58.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.070+0000 7f6f60c00640 1 -- 172.21.15.103:0/1516767435 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c017030 con 0x7f6f6410f760 2024-09-11T00:32:58.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.070+0000 7f6f60c00640 1 -- 172.21.15.103:0/1516767435 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5c004020 con 0x7f6f6410f760 2024-09-11T00:32:58.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.070+0000 7f6f60c00640 1 -- 172.21.15.103:0/1516767435 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c003b50 con 0x7f6f6410f760 2024-09-11T00:32:58.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.070+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f641a5c60 con 0x7f6f6410f760 2024-09-11T00:32:58.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.070+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f641ac320 con 0x7f6f6410f760 2024-09-11T00:32:58.074 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.071+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f641097b0 con 0x7f6f6410f760 2024-09-11T00:32:58.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.074+0000 7f6f60c00640 1 -- 172.21.15.103:0/1516767435 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f6f5c003cf0 con 0x7f6f6410f760 2024-09-11T00:32:58.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.075+0000 7f6f60c00640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6f480b5360 0x7f6f480b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:58.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.076+0000 7f6f60c00640 1 -- 172.21.15.103:0/1516767435 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f6f5c0e5cb0 con 0x7f6f6410f760 2024-09-11T00:32:58.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.076+0000 7f6f63e00640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6f480b5360 0x7f6f480b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:58.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.076+0000 7f6f63e00640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6f480b5360 0x7f6f480b7820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009430 tx=0x7f6f54009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:58.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.077+0000 7f6f60c00640 1 -- 172.21.15.103:0/1516767435 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6f5c021070 con 0x7f6f6410f760 2024-09-11T00:32:58.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.303+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f641a09f0 con 0x7f6f6410f760 2024-09-11T00:32:58.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.303+0000 7f6f60c00640 1 -- 172.21.15.103:0/1516767435 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11937 (secure 0 0 0) 0x7f6f5c0ac800 con 0x7f6f6410f760 2024-09-11T00:32:58.306 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:32:58.306 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":47,"btime":"2024-09-11T00:32:58:220802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:32:57.689346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26323":{"gid":26323,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4278110475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4278110475},{"type":"v1","addr":"172.21.15.183:6835","nonce":4278110475}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:32:58.306 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 47 2024-09-11T00:32:58.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6f480b5360 msgr2=0x7f6f480b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6f480b5360 0x7f6f480b7820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6f54009430 tx=0x7f6f54009360 comp rx=0 tx=0).stop 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6410f760 msgr2=0x7f6f641a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6410f760 0x7f6f641a51a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00d950 tx=0x7f6f5c00de10 comp rx=0 tx=0).stop 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 shutdown_connections 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f6f480b5360 0x7f6f480b7820 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6410f760 0x7f6f641a51a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f6410ede0 0x7f6f6419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 --2- 172.21.15.103:0/1516767435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6f641093d0 0x7f6f6419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 >> 172.21.15.103:0/1516767435 conn(0x7f6f64078a40 msgr2=0x7f6f6407ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 shutdown_connections 2024-09-11T00:32:58.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:32:58.306+0000 7f6f6b200640 1 -- 172.21.15.103:0/1516767435 wait complete. 2024-09-11T00:32:58.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:58 smithi183 ceph-mon[35934]: pgmap v697: 129 pgs: 129 active+clean; 64 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 473 op/s 2024-09-11T00:32:58.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:58 smithi103 ceph-mon[30578]: pgmap v697: 129 pgs: 129 active+clean; 64 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 473 op/s 2024-09-11T00:32:58.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:58 smithi117 ceph-mon[35525]: pgmap v697: 129 pgs: 129 active+clean; 64 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 473 op/s 2024-09-11T00:32:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 -- 172.21.15.103:0/845263758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c081042c0 msgr2=0x7f4c0810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/845263758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c081042c0 0x7f4c0810ac40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009cd0 tx=0x7f4c0002a8e0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 -- 172.21.15.103:0/845263758 shutdown_connections 2024-09-11T00:32:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/845263758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c081042c0 0x7f4c0810ac40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/845263758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c08103900 0x7f4c08103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/845263758 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c081027b0 0x7f4c08102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 -- 172.21.15.103:0/845263758 >> 172.21.15.103:0/845263758 conn(0x7f4c080fe050 msgr2=0x7f4c08100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 -- 172.21.15.103:0/845263758 shutdown_connections 2024-09-11T00:32:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.995+0000 7f4c0dc00640 1 -- 172.21.15.103:0/845263758 wait complete. 2024-09-11T00:32:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 Processor -- start 2024-09-11T00:32:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 -- start start 2024-09-11T00:32:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c081027b0 0x7f4c0819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c08103900 0x7f4c0819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c081042c0 0x7f4c0819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810e420 con 0x7f4c08103900 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810e2a0 con 0x7f4c081027b0 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.996+0000 7f4c0dc00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810e5a0 con 0x7f4c081042c0 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c08103900 0x7f4c0819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c081027b0 0x7f4c0819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c08103900 0x7f4c0819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41886/0 (socket says 172.21.15.103:41886) 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07400640 1 -- 172.21.15.103:0/1963094994 learned_addr learned my addr 172.21.15.103:0/1963094994 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c0d200640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c081042c0 0x7f4c0819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07e00640 1 -- 172.21.15.103:0/1963094994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c081042c0 msgr2=0x7f4c0819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c081042c0 0x7f4c0819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07e00640 1 -- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c08103900 msgr2=0x7f4c0819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c08103900 0x7f4c0819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07e00640 1 -- 172.21.15.103:0/1963094994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0819c870 con 0x7f4c081027b0 2024-09-11T00:32:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07400640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c08103900 0x7f4c0819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.997+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c081027b0 0x7f4c0819ad30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9f0 tx=0x7f4bf800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.998+0000 7f4c04c00640 1 -- 172.21.15.103:0/1963094994 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800ccc0 con 0x7f4c081027b0 2024-09-11T00:32:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.998+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0819cb60 con 0x7f4c081027b0 2024-09-11T00:32:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.998+0000 7f4c04c00640 1 -- 172.21.15.103:0/1963094994 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bf8005020 con 0x7f4c081027b0 2024-09-11T00:32:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.998+0000 7f4c04c00640 1 -- 172.21.15.103:0/1963094994 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8004a00 con 0x7f4c081027b0 2024-09-11T00:32:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:58.998+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a3230 con 0x7f4c081027b0 2024-09-11T00:32:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 Processor -- start 2024-09-11T00:32:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 -- start start 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6307d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf630d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564bf62786b0 con 0x564bf63082a0 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564bf6267ae0 con 0x564bf6305970 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c12206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bf626b8f0 con 0x564bf630ace0 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.001+0000 7f4c0d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6307d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6307d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46474/0 (socket says 172.21.15.103:46474) 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 -- 172.21.15.103:0/781075362 learned_addr learned my addr 172.21.15.103:0/781075362 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c04c00640 1 -- 172.21.15.103:0/1963094994 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4bf8004ba0 con 0x7f4c081027b0 2024-09-11T00:32:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c07400640 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf630d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 -- 172.21.15.103:0/781075362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 msgr2=0x564bf630d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf630d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 -- 172.21.15.103:0/781075362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 msgr2=0x564bf630a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 -- 172.21.15.103:0/781075362 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf6234780 con 0x564bf6305970 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c07400640 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf630d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.002+0000 7f4c0d200640 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6307d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002ae00 tx=0x7f4c00009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b503df7f4f3364 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4bfd600640 1 -- 172.21.15.103:0/781075362 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00031430 con 0x564bf6305970 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4bfd600640 1 -- 172.21.15.103:0/781075362 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00036040 con 0x564bf6305970 2024-09-11T00:32:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4bfd600640 1 -- 172.21.15.103:0/781075362 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c000042d0 con 0x564bf6305970 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 -- 172.21.15.103:0/781075362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 msgr2=0x564bf6307d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6307d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4c0002ae00 tx=0x7f4c00009a20 comp rx=0 tx=0).stop 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 -- 172.21.15.103:0/781075362 shutdown_connections 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf630d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 --2- 172.21.15.103:0/781075362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6307d60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 -- 172.21.15.103:0/781075362 >> 172.21.15.103:0/781075362 conn(0x564bf6300ac0 msgr2=0x564bf6301850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c04c00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4bec0b52e0 0x7f4bec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.003+0000 7f4c12206740 1 -- 172.21.15.103:0/781075362 shutdown_connections 2024-09-11T00:32:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.004+0000 7f4c12206740 1 -- 172.21.15.103:0/781075362 wait complete. 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.004+0000 7f4c04c00640 1 -- 172.21.15.103:0/1963094994 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4bf80e5f70 con 0x7f4c081027b0 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.004+0000 7f4c07400640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4bec0b52e0 0x7f4bec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.004+0000 7f4c12206740 1 -- start start 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.004+0000 7f4c12206740 1 Processor -- start 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.004+0000 7f4c07400640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4bec0b52e0 0x7f4bec0b77a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40124a0 tx=0x7f4bf4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c12206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf6305970 0x564bf630f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c12206740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c12206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf630ace0 0x564bf6310290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c12206740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564bf619fab0 con 0x564bf63082a0 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c12206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x564bf620e570 con 0x564bf630ace0 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c12206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564bf630dc00 con 0x564bf6305970 2024-09-11T00:32:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c12206740 10 client.0 _ll_register_callbacks cb 0x564bf621d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c07400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c0d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf630ace0 0x564bf6310290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c07400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41908/0 (socket says 172.21.15.103:41908) 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c07400640 1 -- 172.21.15.103:0/2113419624 learned_addr learned my addr 172.21.15.103:0/2113419624 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c07e00640 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf6305970 0x564bf630f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c0d200640 1 -- 172.21.15.103:0/2113419624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf6305970 msgr2=0x564bf630f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c0d200640 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf6305970 0x564bf630f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c0d200640 1 -- 172.21.15.103:0/2113419624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 msgr2=0x564bf630f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c0d200640 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.005+0000 7f4c0d200640 1 -- 172.21.15.103:0/2113419624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf5fc9790 con 0x564bf630ace0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4c0d200640 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf630ace0 0x564bf6310290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009ca0 tx=0x7f4c000098d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadd76aa37c00798 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4bf2000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4bf2000640 1 -- 172.21.15.103:0/2113419624 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c000040b0 con 0x564bf630ace0 2024-09-11T00:32:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c00036040 con 0x564bf630ace0 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4bf2000640 1 -- 172.21.15.103:0/2113419624 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4c00041040 con 0x564bf630ace0 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4bf2000640 1 -- 172.21.15.103:0/2113419624 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4c0005b050 con 0x564bf630ace0 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4c12206740 10 client.26438 fetch_fsmap learned FSMap version 47 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.006+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564bf6042620 con 0x564bf630ace0 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4bf2000640 1 -- 172.21.15.103:0/2113419624 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c00036040 con 0x564bf630ace0 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4bf2000640 1 -- 172.21.15.103:0/2113419624 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f4c00030070 con 0x564bf630ace0 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4c12206740 10 client.26438 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4c12206740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4c12206740 20 client.26438 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4c12206740 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf631a560 0x564bf631c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4c12206740 4 client.26438 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bf627a600 con 0x564bf631a560 2024-09-11T00:32:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.007+0000 7f4c07e00640 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf631a560 0x564bf631c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.008+0000 7f4c07e00640 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf631a560 0x564bf631c950 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32c3b2dad1187bad server_cookie=5db2dc119c5239ea in_seq=0 out_seq=0 2024-09-11T00:32:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.008+0000 7f4bf2000640 10 client.26438 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.008+0000 7f4bf2000640 1 -- 172.21.15.103:0/2113419624 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564bf627a600 con 0x564bf631a560 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.008+0000 7f4bf2000640 10 client.26438 handle_command_reply: tid=0 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.008+0000 7f4c12206740 1 client.26438 shutdown 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.008+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf630ace0 msgr2=0x564bf6310290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.008+0000 7f4c12206740 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf630ace0 0x564bf6310290 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009ca0 tx=0x7f4c000098d0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 shutdown_connections 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf630ace0 0x564bf6310290 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf630f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf631a560 0x564bf631c950 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 --2- 172.21.15.103:0/2113419624 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf6305970 0x564bf630f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4bf2000640 0 client.26438 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 >> 172.21.15.103:0/2113419624 conn(0x564bf6300ac0 msgr2=0x564bf62ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 shutdown_connections 2024-09-11T00:32:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 1 -- 172.21.15.103:0/2113419624 wait complete. 2024-09-11T00:32:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.009+0000 7f4c12206740 20 client.26438 trim_cache size 0 max 16384 2024-09-11T00:32:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 Processor -- start 2024-09-11T00:32:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 -- start start 2024-09-11T00:32:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf62fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf6303900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf6303e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081afda0 con 0x564bf63082a0 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081ad640 con 0x564bf6305970 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.024+0000 7f4c12206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081ab490 con 0x564bf630ace0 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf62fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf62fbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46488/0 (socket says 172.21.15.103:46488) 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 -- 172.21.15.103:0/1415213232 learned_addr learned my addr 172.21.15.103:0/1415213232 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c0d200640 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf6303e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07400640 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf6303900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 -- 172.21.15.103:0/1415213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 msgr2=0x564bf6303e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf6303e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 -- 172.21.15.103:0/1415213232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 msgr2=0x564bf6303900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf6303900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 -- 172.21.15.103:0/1415213232 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf63080d0 con 0x564bf6305970 2024-09-11T00:32:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07400640 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf6303900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:32:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf62fbac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8004770 tx=0x7f4bf8009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b78487edde31e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4bfcc00640 1 -- 172.21.15.103:0/1415213232 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf801c040 con 0x564bf6305970 2024-09-11T00:32:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4bfcc00640 1 -- 172.21.15.103:0/1415213232 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bf80b5210 con 0x564bf6305970 2024-09-11T00:32:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.025+0000 7f4bfcc00640 1 -- 172.21.15.103:0/1415213232 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80b58d0 con 0x564bf6305970 2024-09-11T00:32:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 -- 172.21.15.103:0/1415213232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 msgr2=0x564bf62fbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf62fbac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8004770 tx=0x7f4bf8009f60 comp rx=0 tx=0).stop 2024-09-11T00:32:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 -- 172.21.15.103:0/1415213232 shutdown_connections 2024-09-11T00:32:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf630ace0 0x564bf6303e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf63082a0 0x564bf6303900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 --2- 172.21.15.103:0/1415213232 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf62fbac0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 -- 172.21.15.103:0/1415213232 >> 172.21.15.103:0/1415213232 conn(0x564bf6300ac0 msgr2=0x564bf630b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 -- 172.21.15.103:0/1415213232 shutdown_connections 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 -- 172.21.15.103:0/1415213232 wait complete. 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.026+0000 7f4c12206740 1 -- start start 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 1 Processor -- start 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6316430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf63082a0 0x564bf63169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf630ace0 0x564bf6314d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081abb00 con 0x564bf630ace0 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081acf70 con 0x564bf6305970 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081ac1d0 con 0x564bf63082a0 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c12206740 10 client.0 _ll_register_callbacks cb 0x7f4c00009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c07400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf63082a0 0x564bf63169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c0d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf630ace0 0x564bf6314d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c07400640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf63082a0 0x564bf63169f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:40968/0 (socket says 172.21.15.103:40968) 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c07400640 1 -- 172.21.15.103:0/1254089833 learned_addr learned my addr 172.21.15.103:0/1254089833 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.027+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6316430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c07e00640 1 -- 172.21.15.103:0/1254089833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf63082a0 msgr2=0x564bf63169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf63082a0 0x564bf63169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c07e00640 1 -- 172.21.15.103:0/1254089833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf630ace0 msgr2=0x564bf6314d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf630ace0 0x564bf6314d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c07e00640 1 -- 172.21.15.103:0/1254089833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf6315440 con 0x564bf6305970 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c0d200640 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf630ace0 0x564bf6314d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c07e00640 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6316430 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9c0 tx=0x7f4bf80b0e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215764215b807a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4bf2000640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:32:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4bf2000640 1 -- 172.21.15.103:0/1254089833 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf801c070 con 0x564bf6305970 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564bf62760b0 con 0x564bf6305970 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4bf2000640 1 -- 172.21.15.103:0/1254089833 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4bf80d3180 con 0x564bf6305970 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4bf2000640 1 -- 172.21.15.103:0/1254089833 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4bf80aca40 con 0x564bf6305970 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c12206740 10 client.26444 fetch_fsmap learned FSMap version 47 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.028+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x564bf6387e30 con 0x564bf6305970 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4bf2000640 1 -- 172.21.15.103:0/1254089833 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564bf62760b0 con 0x564bf6305970 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4bf2000640 1 -- 172.21.15.103:0/1254089833 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f4bf80b5440 con 0x564bf6305970 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4c12206740 10 client.26444 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4c12206740 10 client.26444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4c12206740 20 client.26444 populate_metadata read hostname 'smithi103' 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4c12206740 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf6388790 0x564bf638abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4c12206740 4 client.26444 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bf62734e0 con 0x564bf6388790 2024-09-11T00:32:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.029+0000 7f4c07400640 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf6388790 0x564bf638abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:32:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.030+0000 7f4c07400640 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf6388790 0x564bf638abb0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b4e306d67b6c5 server_cookie=3192963fd742e4a6 in_seq=0 out_seq=0 2024-09-11T00:32:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.030+0000 7f4bf2000640 10 client.26444 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4bf2000640 1 -- 172.21.15.103:0/1254089833 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bf62734e0 con 0x564bf6388790 2024-09-11T00:32:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4bf2000640 10 client.26444 handle_command_reply: tid=0 2024-09-11T00:32:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 client.26444 shutdown 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 msgr2=0x564bf6316430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6316430 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9c0 tx=0x7f4bf80b0e90 comp rx=0 tx=0).stop 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 shutdown_connections 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564bf630ace0 0x564bf6314d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x564bf6388790 0x564bf638abb0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564bf63082a0 0x564bf63169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4bf2000640 0 client.26444 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 --2- 172.21.15.103:0/1254089833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x564bf6305970 0x564bf6316430 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 >> 172.21.15.103:0/1254089833 conn(0x564bf6300ac0 msgr2=0x564bf62ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 shutdown_connections 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 1 -- 172.21.15.103:0/1254089833 wait complete. 2024-09-11T00:32:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.045+0000 7f4c12206740 20 client.26444 trim_cache size 0 max 16384 2024-09-11T00:32:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:{ 2024-09-11T00:32:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "message": "", 2024-09-11T00:32:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout: "return_code": 0 2024-09-11T00:32:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stdout:} 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4bec0b52e0 msgr2=0x7f4bec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4bec0b52e0 0x7f4bec0b77a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4bf40124a0 tx=0x7f4bf4005f90 comp rx=0 tx=0).stop 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c081027b0 msgr2=0x7f4c0819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c081027b0 0x7f4c0819ad30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800e9f0 tx=0x7f4bf800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 shutdown_connections 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4bec0b52e0 0x7f4bec0b77a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c081042c0 0x7f4c0819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c08103900 0x7f4c0819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 --2- 172.21.15.103:0/1963094994 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c081027b0 0x7f4c0819ad30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:32:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 >> 172.21.15.103:0/1963094994 conn(0x7f4c080fe050 msgr2=0x7f4c08075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:32:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 shutdown_connections 2024-09-11T00:32:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi103.front.sepia.ceph.com.smithi103.stderr:2024-09-11T00:32:59.046+0000 7f4c0dc00640 1 -- 172.21.15.103:0/1963094994 wait complete. 2024-09-11T00:32:59.212 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24656 quiesce db --cancel --all' 2024-09-11T00:32:59.212 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24656 quiesce db --cancel --all 2024-09-11T00:32:59.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:59 smithi183 ceph-mon[35934]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:59.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:32:59 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1516767435' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:59.626 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:32:59.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:59 smithi103 ceph-mon[30578]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:59.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:32:59 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1516767435' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:32:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:59 smithi117 ceph-mon[35525]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-11T00:32:59.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:32:59 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1516767435' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:33:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 -- 172.21.15.103:0/3236260280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010a800 msgr2=0x7f002010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 --2- 172.21.15.103:0/3236260280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010a800 0x7f002010ac40 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f000c009900 tx=0x7f000c02a410 comp rx=0 tx=0).stop 2024-09-11T00:33:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 -- 172.21.15.103:0/3236260280 shutdown_connections 2024-09-11T00:33:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 --2- 172.21.15.103:0/3236260280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f002010b180 0x7f0020111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 --2- 172.21.15.103:0/3236260280 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010a800 0x7f002010ac40 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 --2- 172.21.15.103:0/3236260280 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0020104df0 0x7f00201051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.235 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 -- 172.21.15.103:0/3236260280 >> 172.21.15.103:0/3236260280 conn(0x7f0020100840 msgr2=0x7f0020102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 -- 172.21.15.103:0/3236260280 shutdown_connections 2024-09-11T00:33:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.232+0000 7f0026600640 1 -- 172.21.15.103:0/3236260280 wait complete. 2024-09-11T00:33:00.236 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.233+0000 7f0026600640 1 Processor -- start 2024-09-11T00:33:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.233+0000 7f0026600640 1 -- start start 2024-09-11T00:33:00.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0026600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0020104df0 0x7f002019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0026600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f002010a800 0x7f002019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0026600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010b180 0x7f00201a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0026600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0020114e40 con 0x7f002010b180 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0026600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0020114cc0 con 0x7f0020104df0 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0026600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0020114fc0 con 0x7f002010a800 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010b180 0x7f00201a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0020104df0 0x7f002019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0020104df0 0x7f002019b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46518/0 (socket says 172.21.15.103:46518) 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 -- 172.21.15.103:0/1045187373 learned_addr learned my addr 172.21.15.103:0/1045187373 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f001fe00640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f002010a800 0x7f002019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 -- 172.21.15.103:0/1045187373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f002010a800 msgr2=0x7f002019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f002010a800 0x7f002019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 -- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010b180 msgr2=0x7f00201a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010b180 0x7f00201a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.234+0000 7f0025200640 1 -- 172.21.15.103:0/1045187373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00201a12c0 con 0x7f0020104df0 2024-09-11T00:33:00.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.235+0000 7f0025200640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0020104df0 0x7f002019b210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f001400e9f0 tx=0x7f001400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.235+0000 7f001d600640 1 -- 172.21.15.103:0/1045187373 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001400cd80 con 0x7f0020104df0 2024-09-11T00:33:00.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.235+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00201a15b0 con 0x7f0020104df0 2024-09-11T00:33:00.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.235+0000 7f001d600640 1 -- 172.21.15.103:0/1045187373 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00140054d0 con 0x7f0020104df0 2024-09-11T00:33:00.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.235+0000 7f001d600640 1 -- 172.21.15.103:0/1045187373 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0014010690 con 0x7f0020104df0 2024-09-11T00:33:00.239 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.235+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00201a7c70 con 0x7f0020104df0 2024-09-11T00:33:00.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.238+0000 7f002a7f2740 1 Processor -- start 2024-09-11T00:33:00.240 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.238+0000 7f002a7f2740 1 -- start start 2024-09-11T00:33:00.241 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24c250 0x560a9d24c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24cb70 0x560a9d2585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f002a7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a9d1c3ff0 con 0x560a9d258b00 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f002a7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560a9d1b3420 con 0x560a9d24cb70 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f002a7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a9d1b7230 con 0x560a9d24c250 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f001fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f0025c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24c250 0x560a9d24c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f0025c00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24c250 0x560a9d24c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:41000/0 (socket says 172.21.15.103:41000) 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f0025c00640 1 -- 172.21.15.103:0/2647374771 learned_addr learned my addr 172.21.15.103:0/2647374771 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f001d600640 1 -- 172.21.15.103:0/1045187373 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f0014004e90 con 0x7f0020104df0 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.239+0000 7f0025c00640 1 -- 172.21.15.103:0/2647374771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24cb70 msgr2=0x560a9d2585c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0025c00640 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24cb70 0x560a9d2585c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.242 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0025c00640 1 -- 172.21.15.103:0/2647374771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 msgr2=0x560a9d25aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:33:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0025c00640 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0025c00640 1 -- 172.21.15.103:0/2647374771 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a9cf118b0 con 0x560a9d24c250 2024-09-11T00:33:00.243 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f001fe00640 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0025c00640 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24c250 0x560a9d24c630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0018003a20 tx=0x7f0018003ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d21dfcfc1538841 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0012000640 1 -- 172.21.15.103:0/2647374771 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f001801a470 con 0x560a9d24c250 2024-09-11T00:33:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0012000640 1 -- 172.21.15.103:0/2647374771 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0018015070 con 0x560a9d24c250 2024-09-11T00:33:00.244 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.240+0000 7f0012000640 1 -- 172.21.15.103:0/2647374771 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00180119e0 con 0x560a9d24c250 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f002a7f2740 1 -- 172.21.15.103:0/2647374771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24c250 msgr2=0x560a9d24c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24c250 0x560a9d24c630 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0018003a20 tx=0x7f0018003ee0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f001d600640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f00080b5390 0x7f00080b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f002a7f2740 1 -- 172.21.15.103:0/2647374771 shutdown_connections 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24cb70 0x560a9d2585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2647374771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24c250 0x560a9d24c630 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f002a7f2740 1 -- 172.21.15.103:0/2647374771 >> 172.21.15.103:0/2647374771 conn(0x560a9d24a080 msgr2=0x560a9d24b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.241+0000 7f001d600640 1 -- 172.21.15.103:0/1045187373 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f00140e6340 con 0x7f0020104df0 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.242+0000 7f002a7f2740 1 -- 172.21.15.103:0/2647374771 shutdown_connections 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.242+0000 7f0025c00640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f00080b5390 0x7f00080b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.242+0000 7f002a7f2740 1 -- 172.21.15.103:0/2647374771 wait complete. 2024-09-11T00:33:00.245 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.242+0000 7f002a7f2740 1 -- start start 2024-09-11T00:33:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.242+0000 7f0025c00640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f00080b5390 0x7f00080b7850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0018004010 tx=0x7f0018011440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.246 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.242+0000 7f002a7f2740 1 Processor -- start 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d2584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d255900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d255fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f002a7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a9d014810 con 0x560a9d258b00 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f002a7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x560a9d14e550 con 0x560a9d24c250 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f002a7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560a9d25b7b0 con 0x560a9d24cb70 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f002a7f2740 10 client.0 _ll_register_callbacks cb 0x560a9d15e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f0025200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d2584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f0025c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d255fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f0025200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d2584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46550/0 (socket says 172.21.15.103:46550) 2024-09-11T00:33:00.248 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f0025200640 1 -- 172.21.15.103:0/2254557533 learned_addr learned my addr 172.21.15.103:0/2254557533 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.243+0000 7f001fe00640 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d255900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0025200640 1 -- 172.21.15.103:0/2254557533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 msgr2=0x560a9d255900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0025200640 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d255900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0025200640 1 -- 172.21.15.103:0/2254557533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 msgr2=0x560a9d255fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0025200640 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d255fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0025200640 1 -- 172.21.15.103:0/2254557533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9d140200 con 0x560a9d24c250 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0025200640 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d2584b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f00140097d0 tx=0x7f00140ac600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=336cd7120f851940 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0006a00640 10 client.0 ms_handle_connect on v2:172.21.15.117:3300/0 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0006a00640 1 -- 172.21.15.103:0/2254557533 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140d3020 con 0x560a9d24c250 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0006a00640 1 -- 172.21.15.103:0/2254557533 <== mon.2 v2:172.21.15.117:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f00140b5870 con 0x560a9d24c250 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0018015070 con 0x560a9d24c250 2024-09-11T00:33:00.249 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f0006a00640 1 -- 172.21.15.103:0/2254557533 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f00140b1020 con 0x560a9d24c250 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f002a7f2740 10 client.26453 fetch_fsmap learned FSMap version 47 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.244+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560a9d267220 con 0x560a9d24c250 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f0006a00640 1 -- 172.21.15.103:0/2254557533 <== mon.2 v2:172.21.15.117:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0018015070 con 0x560a9d24c250 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f0006a00640 1 -- 172.21.15.103:0/2254557533 <== mon.2 v2:172.21.15.117:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f00140c3030 con 0x560a9d24c250 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f002a7f2740 10 client.26453 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f002a7f2740 10 client.26453 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f002a7f2740 20 client.26453 populate_metadata read hostname 'smithi103' 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2686e0 0x560a9d26aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f002a7f2740 4 client.26453 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T00:33:00.250 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.245+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a9d1c5f40 con 0x560a9d2686e0 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.246+0000 7f001fe00640 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2686e0 0x560a9d26aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.246+0000 7f001fe00640 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2686e0 0x560a9d26aad0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64f1deeef336067 server_cookie=70211c4a5b0e084c in_seq=0 out_seq=0 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.246+0000 7f0006a00640 10 client.26453 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f0006a00640 1 -- 172.21.15.103:0/2254557533 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a9d1c5f40 con 0x560a9d2686e0 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f0006a00640 10 client.26453 handle_command_reply: tid=0 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 client.26453 shutdown 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 msgr2=0x560a9d2584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d2584b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f00140097d0 tx=0x7f00140ac600 comp rx=0 tx=0).stop 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 shutdown_connections 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d255fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2686e0 0x560a9d26aad0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.252 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d255900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2254557533 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d2584b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f0006a00640 0 client.26453 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:33:00.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.247+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 >> 172.21.15.103:0/2254557533 conn(0x560a9d24a080 msgr2=0x560a9d24fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:00.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.248+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 shutdown_connections 2024-09-11T00:33:00.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.248+0000 7f002a7f2740 1 -- 172.21.15.103:0/2254557533 wait complete. 2024-09-11T00:33:00.253 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.248+0000 7f002a7f2740 20 client.26453 trim_cache size 0 max 16384 2024-09-11T00:33:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 Processor -- start 2024-09-11T00:33:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 -- start start 2024-09-11T00:33:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d249030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d259ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.268 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f00201b42b0 con 0x560a9d258b00 2024-09-11T00:33:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00201b1b50 con 0x560a9d24c250 2024-09-11T00:33:00.269 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f002a7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00201af9a0 con 0x560a9d24cb70 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f0025c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.266+0000 7f001fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d259ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d249030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f001fe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d259ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:41022/0 (socket says 172.21.15.103:41022) 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41984/0 (socket says 172.21.15.103:41984) 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f001fe00640 1 -- 172.21.15.103:0/4000522152 learned_addr learned my addr 172.21.15.103:0/4000522152 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025200640 1 -- 172.21.15.103:0/4000522152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 msgr2=0x560a9d259ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025200640 1 --2- 172.21.15.103:0/4000522152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d259ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025200640 1 -- 172.21.15.103:0/4000522152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 msgr2=0x560a9d25a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:33:00.270 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025200640 1 --2- 172.21.15.103:0/4000522152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025200640 1 -- 172.21.15.103:0/4000522152 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a9d26b100 con 0x560a9d24c250 2024-09-11T00:33:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0025200640 1 --2- 172.21.15.103:0/4000522152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d249030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f00140b5d00 tx=0x7f00140ac670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2753511300c61de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.267+0000 7f0011600640 1 -- 172.21.15.103:0/4000522152 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140d3040 con 0x560a9d24c250 2024-09-11T00:33:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f0011600640 1 -- 172.21.15.103:0/4000522152 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00140e6170 con 0x560a9d24c250 2024-09-11T00:33:00.271 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f0011600640 1 -- 172.21.15.103:0/4000522152 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00140b5ac0 con 0x560a9d24c250 2024-09-11T00:33:00.272 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 -- 172.21.15.103:0/4000522152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 msgr2=0x560a9d249030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 --2- 172.21.15.103:0/4000522152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d249030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f00140b5d00 tx=0x7f00140ac670 comp rx=0 tx=0).stop 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 -- 172.21.15.103:0/4000522152 shutdown_connections 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 --2- 172.21.15.103:0/4000522152 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d258b00 0x560a9d25a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 --2- 172.21.15.103:0/4000522152 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d24cb70 0x560a9d259ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 --2- 172.21.15.103:0/4000522152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d249030 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 -- 172.21.15.103:0/4000522152 >> 172.21.15.103:0/4000522152 conn(0x560a9d24a080 msgr2=0x560a9d248920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.268+0000 7f002a7f2740 1 -- 172.21.15.103:0/4000522152 shutdown_connections 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.269+0000 7f002a7f2740 1 -- 172.21.15.103:0/4000522152 wait complete. 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.269+0000 7f002a7f2740 1 -- start start 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.269+0000 7f002a7f2740 1 Processor -- start 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d260a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.273 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d24cb70 0x560a9d260f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f002a7f2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d258b00 0x560a9d252130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f002a7f2740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f00201b0010 con 0x560a9d24cb70 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f002a7f2740 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f00201b1480 con 0x560a9d24c250 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f002a7f2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00201b06e0 con 0x560a9d258b00 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f002a7f2740 10 client.0 _ll_register_callbacks cb 0x7f00140a8610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d24cb70 0x560a9d260f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d24cb70 0x560a9d260f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:41998/0 (socket says 172.21.15.103:41998) 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 -- 172.21.15.103:0/2788870764 learned_addr learned my addr 172.21.15.103:0/2788870764 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f0025200640 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d260a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f0025c00640 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d258b00 0x560a9d252130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 -- 172.21.15.103:0/2788870764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d258b00 msgr2=0x560a9d252130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d258b00 0x560a9d252130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 -- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 msgr2=0x560a9d260a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T00:33:00.274 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d260a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.270+0000 7f001fe00640 1 -- 172.21.15.103:0/2788870764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9d2526f0 con 0x560a9d24cb70 2024-09-11T00:33:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f001fe00640 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d24cb70 0x560a9d260f80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f000c02a930 tx=0x7f000c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51eac76ad1eac08e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:00.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f0006a00640 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f0006a00640 1 -- 172.21.15.103:0/2788870764 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c035040 con 0x560a9d24cb70 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00140e6170 con 0x560a9d24cb70 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f0006a00640 1 -- 172.21.15.103:0/2788870764 <== mon.0 v2:172.21.15.103:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f000c005450 con 0x560a9d24cb70 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f0006a00640 1 -- 172.21.15.103:0/2788870764 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f000c057050 con 0x560a9d24cb70 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f002a7f2740 10 client.16383 fetch_fsmap learned FSMap version 47 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.271+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x560a9d2d6e70 con 0x560a9d24cb70 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f0006a00640 1 -- 172.21.15.103:0/2788870764 <== mon.0 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00140e6170 con 0x560a9d24cb70 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f0006a00640 1 -- 172.21.15.103:0/2788870764 <== mon.0 v2:172.21.15.103:3300/0 5 ==== fsmap(e 47) ==== 8366+0+0 (secure 0 0 0) 0x7f00140051b0 con 0x560a9d24cb70 2024-09-11T00:33:00.277 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f002a7f2740 10 client.16383 fetch_fsmap finished waiting for FSMap version 47 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f002a7f2740 10 client.16383 resolve_mds: validated gid 24656 aka daemon mds.b 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f002a7f2740 20 client.16383 populate_metadata read hostname 'smithi103' 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2d7570 0x560a9d2d9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f002a7f2740 4 client.16383 mds_command: new command op to 24656 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 --> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x560a9d1c19f0 con 0x560a9d2d7570 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.272+0000 7f0025200640 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2d7570 0x560a9d2d9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f0025200640 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2d7570 0x560a9d2d9990 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d926579b67a80e server_cookie=7ad9753cc9dbe0e4 in_seq=0 out_seq=0 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f0006a00640 10 client.16383 ms_handle_connect on v2:172.21.15.117:6838/3769588373 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f0006a00640 1 -- 172.21.15.103:0/2788870764 <== mds.0 v2:172.21.15.117:6838/3769588373 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x560a9d1c19f0 con 0x560a9d2d7570 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f0006a00640 10 client.16383 handle_command_reply: tid=0 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 client.16383 shutdown 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d24cb70 msgr2=0x560a9d260f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d24cb70 0x560a9d260f80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f000c02a930 tx=0x7f000c009680 comp rx=0 tx=0).stop 2024-09-11T00:33:00.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 shutdown_connections 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:6838/3769588373,v1:172.21.15.117:6839/3769588373] conn(0x560a9d2d7570 0x560a9d2d9990 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560a9d258b00 0x560a9d252130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9d24cb70 0x560a9d260f80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 --2- 172.21.15.103:0/2788870764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x560a9d24c250 0x560a9d260a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 >> 172.21.15.103:0/2788870764 conn(0x560a9d24a080 msgr2=0x560a9d249140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.273+0000 7f0006a00640 0 client.16383 ms_handle_reset on v2:172.21.15.117:6838/3769588373 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 shutdown_connections 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f002a7f2740 1 -- 172.21.15.103:0/2788870764 wait complete. 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f002a7f2740 20 client.16383 trim_cache size 0 max 16384 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:33:00.279 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 47, 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stdout: "leader": 24656, 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stdout: "set_version": 28, 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stdout: "sets": {} 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f00080b5390 msgr2=0x7f00080b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f00080b5390 0x7f00080b7850 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0018004010 tx=0x7f0018011440 comp rx=0 tx=0).stop 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0020104df0 msgr2=0x7f002019b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:00.280 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0020104df0 0x7f002019b210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f001400e9f0 tx=0x7f001400edf0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 shutdown_connections 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f00080b5390 0x7f00080b7850 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f002010b180 0x7f00201a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f002010a800 0x7f002019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 --2- 172.21.15.103:0/1045187373 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0020104df0 0x7f002019b210 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.274+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 >> 172.21.15.103:0/1045187373 conn(0x7f0020100840 msgr2=0x7f0020102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.275+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 shutdown_connections 2024-09-11T00:33:00.281 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:00.275+0000 7f0026600640 1 -- 172.21.15.103:0/1045187373 wait complete. 2024-09-11T00:33:00.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:00 smithi103 ceph-mon[30578]: pgmap v698: 129 pgs: 129 active+clean; 60 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 544 op/s 2024-09-11T00:33:00.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:00 smithi183 ceph-mon[35934]: pgmap v698: 129 pgs: 129 active+clean; 60 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 544 op/s 2024-09-11T00:33:00.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:00 smithi117 ceph-mon[35525]: pgmap v698: 129 pgs: 129 active+clean; 60 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 544 op/s 2024-09-11T00:33:01.202 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T00:33:01.202 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T00:33:01.216 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:33:01.228 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T00:33:01.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 -- 172.21.15.103:0/313258642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 msgr2=0x7f9168106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 --2- 172.21.15.103:0/313258642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f9168106170 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9154009930 tx=0x7f915402a420 comp rx=0 tx=0).stop 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 -- 172.21.15.103:0/313258642 shutdown_connections 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 --2- 172.21.15.103:0/313258642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91681066b0 0x7f916810d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 --2- 172.21.15.103:0/313258642 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f9168106170 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 --2- 172.21.15.103:0/313258642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9168104ba0 0x7f9168104fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 -- 172.21.15.103:0/313258642 >> 172.21.15.103:0/313258642 conn(0x7f9168100440 msgr2=0x7f9168102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 -- 172.21.15.103:0/313258642 shutdown_connections 2024-09-11T00:33:01.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.424+0000 7f916de00640 1 -- 172.21.15.103:0/313258642 wait complete. 2024-09-11T00:33:01.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.425+0000 7f916de00640 1 Processor -- start 2024-09-11T00:33:01.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.425+0000 7f916de00640 1 -- start start 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f916de00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168104ba0 0x7f916819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f916de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f916819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f916de00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91681066b0 0x7f916819c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f916de00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9168110860 con 0x7f9168105cf0 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f916de00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f91681106e0 con 0x7f91681066b0 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f916de00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f91681109e0 con 0x7f9168104ba0 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f916819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f916819b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42028/0 (socket says 172.21.15.103:42028) 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166000640 1 -- 172.21.15.103:0/2782965690 learned_addr learned my addr 172.21.15.103:0/2782965690 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166a00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168104ba0 0x7f916819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9167400640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91681066b0 0x7f916819c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166000640 1 -- 172.21.15.103:0/2782965690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168104ba0 msgr2=0x7f916819af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166000640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168104ba0 0x7f916819af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166000640 1 -- 172.21.15.103:0/2782965690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91681066b0 msgr2=0x7f916819c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.426+0000 7f9166000640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91681066b0 0x7f916819c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.427+0000 7f9166000640 1 -- 172.21.15.103:0/2782965690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916819caa0 con 0x7f9168105cf0 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.427+0000 7f9166000640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f916819b5b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9154031500 tx=0x7f9154003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.427+0000 7f916ca00640 1 -- 172.21.15.103:0/2782965690 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915403f070 con 0x7f9168105cf0 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.427+0000 7f916ca00640 1 -- 172.21.15.103:0/2782965690 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9154031c50 con 0x7f9168105cf0 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.427+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916819cd30 con 0x7f9168105cf0 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.427+0000 7f916ca00640 1 -- 172.21.15.103:0/2782965690 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9154035030 con 0x7f9168105cf0 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.427+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91681a3460 con 0x7f9168105cf0 2024-09-11T00:33:01.431 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.428+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9168104fa0 con 0x7f9168105cf0 2024-09-11T00:33:01.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.431+0000 7f916ca00640 1 -- 172.21.15.103:0/2782965690 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f915403b460 con 0x7f9168105cf0 2024-09-11T00:33:01.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.432+0000 7f916ca00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f914c0b5360 0x7f914c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.433+0000 7f916ca00640 1 -- 172.21.15.103:0/2782965690 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f9154106940 con 0x7f9168105cf0 2024-09-11T00:33:01.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.433+0000 7f9166a00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f914c0b5360 0x7f914c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.433+0000 7f9166a00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f914c0b5360 0x7f914c0b7820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9158009bd0 tx=0x7f9158005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:01.439 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.437+0000 7f916ca00640 1 -- 172.21.15.103:0/2782965690 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f91540ccdc0 con 0x7f9168105cf0 2024-09-11T00:33:01.629 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.627+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9168063f10 con 0x7f9168105cf0 2024-09-11T00:33:01.631 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.629+0000 7f916ca00640 1 -- 172.21.15.103:0/2782965690 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24865 (secure 0 0 0) 0x7f91540d17f0 con 0x7f9168105cf0 2024-09-11T00:33:01.631 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:33:01.631 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":92,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:31:53.221970+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.631+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f914c0b5360 msgr2=0x7f914c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.631+0000 7f916de00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f914c0b5360 0x7f914c0b7820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9158009bd0 tx=0x7f9158005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.631+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 msgr2=0x7f916819b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.631+0000 7f916de00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f916819b5b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9154031500 tx=0x7f9154003c10 comp rx=0 tx=0).stop 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 shutdown_connections 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f914c0b5360 0x7f914c0b7820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91681066b0 0x7f916819c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9168105cf0 0x7f916819b5b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 --2- 172.21.15.103:0/2782965690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9168104ba0 0x7f916819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 >> 172.21.15.103:0/2782965690 conn(0x7f9168100440 msgr2=0x7f9168109010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:01.634 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 shutdown_connections 2024-09-11T00:33:01.635 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.632+0000 7f916de00640 1 -- 172.21.15.103:0/2782965690 wait complete. 2024-09-11T00:33:01.650 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T00:15:22.579961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T00:20:22.113281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T00:20:25.223289+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-11T00:20:35.424587+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-11T00:33:01.650 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T00:33:01.809 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 -- 172.21.15.103:0/302502586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 msgr2=0x7fbaec104f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 --2- 172.21.15.103:0/302502586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec104f80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009860 tx=0x7fbadc02a340 comp rx=0 tx=0).stop 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 -- 172.21.15.103:0/302502586 shutdown_connections 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 --2- 172.21.15.103:0/302502586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 0x7fbaec10d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 --2- 172.21.15.103:0/302502586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec105cd0 0x7fbaec106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 --2- 172.21.15.103:0/302502586 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec104f80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 -- 172.21.15.103:0/302502586 >> 172.21.15.103:0/302502586 conn(0x7fbaec100440 msgr2=0x7fbaec102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 -- 172.21.15.103:0/302502586 shutdown_connections 2024-09-11T00:33:01.810 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.807+0000 7fbaf1200640 1 -- 172.21.15.103:0/302502586 wait complete. 2024-09-11T00:33:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.808+0000 7fbaf1200640 1 Processor -- start 2024-09-11T00:33:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.808+0000 7fbaf1200640 1 -- start start 2024-09-11T00:33:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.808+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec105cd0 0x7fbaec19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.812 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaf1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 0x7fbaec19c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaf1200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec1107d0 con 0x7fbaec106690 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaf1200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec110650 con 0x7fbaec104b80 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaf1200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec110950 con 0x7fbaec105cd0 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaea800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 0x7fbaec19c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaea800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 0x7fbaec19c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42046/0 (socket says 172.21.15.103:42046) 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:46598/0 (socket says 172.21.15.103:46598) 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaea800640 1 -- 172.21.15.103:0/866096326 learned_addr learned my addr 172.21.15.103:0/866096326 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9400640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec105cd0 0x7fbaec19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9e00640 1 -- 172.21.15.103:0/866096326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec105cd0 msgr2=0x7fbaec19b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9e00640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec105cd0 0x7fbaec19b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9e00640 1 -- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 msgr2=0x7fbaec19c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:01.813 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9e00640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 0x7fbaec19c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9e00640 1 -- 172.21.15.103:0/866096326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec19ca30 con 0x7fbaec104b80 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbae9400640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec105cd0 0x7fbaec19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.809+0000 7fbaea800640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 0x7fbaec19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.810+0000 7fbae9e00640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec19aef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbadc02a860 tx=0x7fbadc032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.810+0000 7fbae2a00640 1 -- 172.21.15.103:0/866096326 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc037040 con 0x7fbaec104b80 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.810+0000 7fbae2a00640 1 -- 172.21.15.103:0/866096326 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbadc030bc0 con 0x7fbaec104b80 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.810+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec19ccc0 con 0x7fbaec104b80 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.810+0000 7fbae2a00640 1 -- 172.21.15.103:0/866096326 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc003850 con 0x7fbaec104b80 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.810+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec1a3210 con 0x7fbaec104b80 2024-09-11T00:33:01.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.811+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec10be90 con 0x7fbaec104b80 2024-09-11T00:33:01.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.813+0000 7fbae2a00640 1 -- 172.21.15.103:0/866096326 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fbadc030580 con 0x7fbaec104b80 2024-09-11T00:33:01.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.815+0000 7fbae2a00640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbad00b52e0 0x7fbad00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:01.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.815+0000 7fbae2a00640 1 -- 172.21.15.103:0/866096326 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fbadc1057b0 con 0x7fbaec104b80 2024-09-11T00:33:01.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.815+0000 7fbae9400640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbad00b52e0 0x7fbad00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:01.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.815+0000 7fbae9400640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbad00b52e0 0x7fbad00b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009c20 tx=0x7fbad8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:01.819 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.817+0000 7fbae2a00640 1 -- 172.21.15.103:0/866096326 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbadc04a060 con 0x7fbaec104b80 2024-09-11T00:33:02.001 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:01.999+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbaec02e750 con 0x7fbaec104b80 2024-09-11T00:33:02.002 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.000+0000 7fbae2a00640 1 -- 172.21.15.103:0/866096326 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fbadc0cc300 con 0x7fbaec104b80 2024-09-11T00:33:02.002 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 1 2024-09-11T00:33:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbad00b52e0 msgr2=0x7fbad00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbad00b52e0 0x7fbad00b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009c20 tx=0x7fbad8005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 msgr2=0x7fbaec19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec19aef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbadc02a860 tx=0x7fbadc032dc0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 shutdown_connections 2024-09-11T00:33:02.005 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fbad00b52e0 0x7fbad00b77a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbaec106690 0x7fbaec19c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbaec105cd0 0x7fbaec19b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 --2- 172.21.15.103:0/866096326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbaec104b80 0x7fbaec19aef0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 >> 172.21.15.103:0/866096326 conn(0x7fbaec100440 msgr2=0x7fbaec109060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:02.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 shutdown_connections 2024-09-11T00:33:02.006 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.003+0000 7fbaf1200640 1 -- 172.21.15.103:0/866096326 wait complete. 2024-09-11T00:33:02.021 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T00:33:02.180 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 -- 172.21.15.103:0/3909749346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98106690 msgr2=0x7f4e9810d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 --2- 172.21.15.103:0/3909749346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98106690 0x7f4e9810d010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009900 tx=0x7f4e9002a410 comp rx=0 tx=0).stop 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 -- 172.21.15.103:0/3909749346 shutdown_connections 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 --2- 172.21.15.103:0/3909749346 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98106690 0x7f4e9810d010 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 --2- 172.21.15.103:0/3909749346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e98105cd0 0x7f4e98106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 --2- 172.21.15.103:0/3909749346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e98104b80 0x7f4e98104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 -- 172.21.15.103:0/3909749346 >> 172.21.15.103:0/3909749346 conn(0x7f4e98100440 msgr2=0x7f4e98102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 -- 172.21.15.103:0/3909749346 shutdown_connections 2024-09-11T00:33:02.181 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.178+0000 7f4e9d200640 1 -- 172.21.15.103:0/3909749346 wait complete. 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 Processor -- start 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 -- start start 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e98104b80 0x7f4e9819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98105cd0 0x7f4e9819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e98106690 0x7f4e9819c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e981107f0 con 0x7f4e98106690 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98110670 con 0x7f4e98105cd0 2024-09-11T00:33:02.183 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.179+0000 7f4e9d200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98110970 con 0x7f4e98104b80 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e96800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e98106690 0x7f4e9819c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98105cd0 0x7f4e9819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98105cd0 0x7f4e9819b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:40270/0 (socket says 172.21.15.103:40270) 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 -- 172.21.15.103:0/4240189071 learned_addr learned my addr 172.21.15.103:0/4240189071 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e95e00640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e98104b80 0x7f4e9819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 -- 172.21.15.103:0/4240189071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e98104b80 msgr2=0x7f4e9819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e98104b80 0x7f4e9819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 -- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e98106690 msgr2=0x7f4e9819c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e98106690 0x7f4e9819c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 -- 172.21.15.103:0/4240189071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9819ca00 con 0x7f4e98105cd0 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.180+0000 7f4e8d400640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98105cd0 0x7f4e9819b510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000e9f0 tx=0x7f4e8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.181+0000 7f4e8f400640 1 -- 172.21.15.103:0/4240189071 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000cd80 con 0x7f4e98105cd0 2024-09-11T00:33:02.184 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.181+0000 7f4e8f400640 1 -- 172.21.15.103:0/4240189071 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e800108e0 con 0x7f4e98105cd0 2024-09-11T00:33:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.181+0000 7f4e8f400640 1 -- 172.21.15.103:0/4240189071 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80018980 con 0x7f4e98105cd0 2024-09-11T00:33:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.181+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9819ccf0 con 0x7f4e98105cd0 2024-09-11T00:33:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.181+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e981a33c0 con 0x7f4e98105cd0 2024-09-11T00:33:02.185 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.182+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9810be00 con 0x7f4e98105cd0 2024-09-11T00:33:02.186 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.184+0000 7f4e8f400640 1 -- 172.21.15.103:0/4240189071 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f4e80010430 con 0x7f4e98105cd0 2024-09-11T00:33:02.187 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.186+0000 7f4e8f400640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e7c0b52e0 0x7f4e7c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.186+0000 7f4e8f400640 1 -- 172.21.15.103:0/4240189071 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f4e800e67c0 con 0x7f4e98105cd0 2024-09-11T00:33:02.188 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.186+0000 7f4e95e00640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e7c0b52e0 0x7f4e7c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.187+0000 7f4e95e00640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e7c0b52e0 0x7f4e7c0b77a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4e88005fd0 tx=0x7f4e88009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:02.189 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.187+0000 7f4e8f400640 1 -- 172.21.15.103:0/4240189071 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4e800acc40 con 0x7f4e98105cd0 2024-09-11T00:33:02.375 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.373+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e9802e5c0 con 0x7f4e98105cd0 2024-09-11T00:33:02.376 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.374+0000 7f4e8f400640 1 -- 172.21.15.103:0/4240189071 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f4e800b1670 con 0x7f4e98105cd0 2024-09-11T00:33:02.376 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-11T00:33:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e7c0b52e0 msgr2=0x7f4e7c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e7c0b52e0 0x7f4e7c0b77a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4e88005fd0 tx=0x7f4e88009500 comp rx=0 tx=0).stop 2024-09-11T00:33:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98105cd0 msgr2=0x7f4e9819b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.379 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98105cd0 0x7f4e9819b510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000e9f0 tx=0x7f4e8000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 shutdown_connections 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f4e7c0b52e0 0x7f4e7c0b77a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e98106690 0x7f4e9819c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e98105cd0 0x7f4e9819b510 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 --2- 172.21.15.103:0/4240189071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e98104b80 0x7f4e9819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 >> 172.21.15.103:0/4240189071 conn(0x7f4e98100440 msgr2=0x7f4e98108fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 shutdown_connections 2024-09-11T00:33:02.380 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.377+0000 7f4e9d200640 1 -- 172.21.15.103:0/4240189071 wait complete. 2024-09-11T00:33:02.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:02 smithi103 ceph-mon[30578]: pgmap v699: 129 pgs: 129 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 521 op/s 2024-09-11T00:33:02.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:02 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2782965690' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:33:02.393 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:02 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/866096326' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:33:02.395 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T00:33:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:02 smithi183 ceph-mon[35934]: pgmap v699: 129 pgs: 129 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 521 op/s 2024-09-11T00:33:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:02 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2782965690' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:33:02.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:02 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/866096326' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:33:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 -- 172.21.15.103:0/1691503516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c076f30 msgr2=0x7f109c075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 --2- 172.21.15.103:0/1691503516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c076f30 0x7f109c075580 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f108c009900 tx=0x7f108c02a410 comp rx=0 tx=0).stop 2024-09-11T00:33:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 -- 172.21.15.103:0/1691503516 shutdown_connections 2024-09-11T00:33:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 --2- 172.21.15.103:0/1691503516 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c10a8e0 0x7f109c10cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 --2- 172.21.15.103:0/1691503516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f109c075ac0 0x7f109c075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 --2- 172.21.15.103:0/1691503516 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c076f30 0x7f109c075580 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.557 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 -- 172.21.15.103:0/1691503516 >> 172.21.15.103:0/1691503516 conn(0x7f109c0fe220 msgr2=0x7f109c100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 -- 172.21.15.103:0/1691503516 shutdown_connections 2024-09-11T00:33:02.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.555+0000 7f10a2000640 1 -- 172.21.15.103:0/1691503516 wait complete. 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 Processor -- start 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 -- start start 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f109c075ac0 0x7f109c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c076f30 0x7f109c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c10a8e0 0x7f109c19c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f109c110820 con 0x7f109c10a8e0 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f109c1106a0 con 0x7f109c076f30 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f10a2000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f109c1109a0 con 0x7f109c075ac0 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.556+0000 7f109be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c10a8e0 0x7f109c19c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c10a8e0 0x7f109c19c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60322/0 (socket says 172.21.15.103:60322) 2024-09-11T00:33:02.560 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 -- 172.21.15.103:0/812484018 learned_addr learned my addr 172.21.15.103:0/812484018 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109b400640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f109c075ac0 0x7f109c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f1093e00640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c076f30 0x7f109c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 -- 172.21.15.103:0/812484018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f109c075ac0 msgr2=0x7f109c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f109c075ac0 0x7f109c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 -- 172.21.15.103:0/812484018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c076f30 msgr2=0x7f109c19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c076f30 0x7f109c19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 -- 172.21.15.103:0/812484018 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109c19cab0 con 0x7f109c10a8e0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f109be00640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c10a8e0 0x7f109c19c1f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f109400eb10 tx=0x7f109400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f1098c00640 1 -- 172.21.15.103:0/812484018 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094004b20 con 0x7f109c10a8e0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f1098c00640 1 -- 172.21.15.103:0/812484018 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1094004cc0 con 0x7f109c10a8e0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109c19cda0 con 0x7f109c10a8e0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.557+0000 7f1098c00640 1 -- 172.21.15.103:0/812484018 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094010670 con 0x7f109c10a8e0 2024-09-11T00:33:02.561 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.558+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f109c1a3470 con 0x7f109c10a8e0 2024-09-11T00:33:02.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.559+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f109c1a3640 con 0x7f109c10a8e0 2024-09-11T00:33:02.562 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.561+0000 7f1098c00640 1 -- 172.21.15.103:0/812484018 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1094005020 con 0x7f109c10a8e0 2024-09-11T00:33:02.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.562+0000 7f1098c00640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10800b5360 0x7f10800b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.562+0000 7f1098c00640 1 -- 172.21.15.103:0/812484018 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f10940e6500 con 0x7f109c10a8e0 2024-09-11T00:33:02.564 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.562+0000 7f109b400640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10800b5360 0x7f10800b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.565 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.563+0000 7f109b400640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10800b5360 0x7f10800b7820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f108c031500 tx=0x7f108c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:02.567 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.566+0000 7f1098c00640 1 -- 172.21.15.103:0/812484018 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f10940acfd0 con 0x7f109c10a8e0 2024-09-11T00:33:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:02 smithi117 ceph-mon[35525]: pgmap v699: 129 pgs: 129 active+clean; 48 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 521 op/s 2024-09-11T00:33:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:02 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2782965690' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:33:02.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:02 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/866096326' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-11T00:33:02.753 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.751+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f109c1a3810 con 0x7f109c10a8e0 2024-09-11T00:33:02.754 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.752+0000 7f1098c00640 1 -- 172.21.15.103:0/812484018 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f10940ac8d0 con 0x7f109c10a8e0 2024-09-11T00:33:02.754 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 64 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10800b5360 msgr2=0x7f10800b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10800b5360 0x7f10800b7820 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f108c031500 tx=0x7f108c005a20 comp rx=0 tx=0).stop 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c10a8e0 msgr2=0x7f109c19c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c10a8e0 0x7f109c19c1f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f109400eb10 tx=0x7f109400c2d0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 shutdown_connections 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10800b5360 0x7f10800b7820 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f109c10a8e0 0x7f109c19c1f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f109c076f30 0x7f109c19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 --2- 172.21.15.103:0/812484018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f109c075ac0 0x7f109c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 >> 172.21.15.103:0/812484018 conn(0x7f109c0fe220 msgr2=0x7f109c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:02.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 shutdown_connections 2024-09-11T00:33:02.758 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.754+0000 7f10a2000640 1 -- 172.21.15.103:0/812484018 wait complete. 2024-09-11T00:33:02.773 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-11T00:33:02.932 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 -- 172.21.15.103:0/3009100694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c1042c0 msgr2=0x7f5a9c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 --2- 172.21.15.103:0/3009100694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c1042c0 0x7f5a9c10ac40 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5a94009cd0 tx=0x7f5a9402a8e0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 -- 172.21.15.103:0/3009100694 shutdown_connections 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 --2- 172.21.15.103:0/3009100694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c1042c0 0x7f5a9c10ac40 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 --2- 172.21.15.103:0/3009100694 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a9c103900 0x7f5a9c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 --2- 172.21.15.103:0/3009100694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9c1027b0 0x7f5a9c102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 -- 172.21.15.103:0/3009100694 >> 172.21.15.103:0/3009100694 conn(0x7f5a9c0fe050 msgr2=0x7f5a9c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 -- 172.21.15.103:0/3009100694 shutdown_connections 2024-09-11T00:33:02.933 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.930+0000 7f5a9be00640 1 -- 172.21.15.103:0/3009100694 wait complete. 2024-09-11T00:33:02.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 Processor -- start 2024-09-11T00:33:02.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 -- start start 2024-09-11T00:33:02.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9c1027b0 0x7f5a9c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c103900 0x7f5a9c19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a9c1042c0 0x7f5a9c19bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.935 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c10e400 con 0x7f5a9c103900 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c10e280 con 0x7f5a9c1042c0 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.931+0000 7f5a9be00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c10e580 con 0x7f5a9c1027b0 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c103900 0x7f5a9c19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c103900 0x7f5a9c19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:60348/0 (socket says 172.21.15.103:60348) 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 -- 172.21.15.103:0/1779605910 learned_addr learned my addr 172.21.15.103:0/1779605910 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9b400640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a9c1042c0 0x7f5a9c19bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9aa00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9c1027b0 0x7f5a9c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 -- 172.21.15.103:0/1779605910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9c1027b0 msgr2=0x7f5a9c19ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9c1027b0 0x7f5a9c19ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 -- 172.21.15.103:0/1779605910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a9c1042c0 msgr2=0x7f5a9c19bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a9c1042c0 0x7f5a9c19bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 -- 172.21.15.103:0/1779605910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c19c870 con 0x7f5a9c103900 2024-09-11T00:33:02.936 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.932+0000 7f5a9a000640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c103900 0x7f5a9c19b380 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e9f0 tx=0x7f5a8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:02.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.933+0000 7f5aa0a00640 1 -- 172.21.15.103:0/1779605910 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800cd80 con 0x7f5a9c103900 2024-09-11T00:33:02.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.933+0000 7f5aa0a00640 1 -- 172.21.15.103:0/1779605910 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a880108e0 con 0x7f5a9c103900 2024-09-11T00:33:02.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.933+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9c19cb60 con 0x7f5a9c103900 2024-09-11T00:33:02.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.933+0000 7f5aa0a00640 1 -- 172.21.15.103:0/1779605910 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88018980 con 0x7f5a9c103900 2024-09-11T00:33:02.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.933+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9c1a3230 con 0x7f5a9c103900 2024-09-11T00:33:02.937 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.934+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a9c1a3400 con 0x7f5a9c103900 2024-09-11T00:33:02.938 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.936+0000 7f5aa0a00640 1 -- 172.21.15.103:0/1779605910 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f5a88010430 con 0x7f5a9c103900 2024-09-11T00:33:02.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.938+0000 7f5aa0a00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5a800b52e0 0x7f5a800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:02.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.938+0000 7f5aa0a00640 1 -- 172.21.15.103:0/1779605910 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f5a880e67c0 con 0x7f5a9c103900 2024-09-11T00:33:02.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.938+0000 7f5a9aa00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5a800b52e0 0x7f5a800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:02.940 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.939+0000 7f5a9aa00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5a800b52e0 0x7f5a800b77a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009bd0 tx=0x7f5a8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:02.941 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:02.939+0000 7f5aa0a00640 1 -- 172.21.15.103:0/1779605910 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5a880acc40 con 0x7f5a9c103900 2024-09-11T00:33:03.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.125+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5a9c1a35d0 con 0x7f5a9c103900 2024-09-11T00:33:03.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.125+0000 7f5aa0a00640 1 -- 172.21.15.103:0/1779605910 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f5a880b1670 con 0x7f5a9c103900 2024-09-11T00:33:03.128 INFO:teuthology.orchestra.run.smithi103.stdout:pg_num: 32 2024-09-11T00:33:03.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5a800b52e0 msgr2=0x7f5a800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5a800b52e0 0x7f5a800b77a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009bd0 tx=0x7f5a8c005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:03.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c103900 msgr2=0x7f5a9c19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c103900 0x7f5a9c19b380 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800e9f0 tx=0x7f5a8800eeb0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 shutdown_connections 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f5a800b52e0 0x7f5a800b77a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a9c1042c0 0x7f5a9c19bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5a9c103900 0x7f5a9c19b380 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 --2- 172.21.15.103:0/1779605910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a9c1027b0 0x7f5a9c19ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 >> 172.21.15.103:0/1779605910 conn(0x7f5a9c0fe050 msgr2=0x7f5a9c075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 shutdown_connections 2024-09-11T00:33:03.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.128+0000 7f5a9be00640 1 -- 172.21.15.103:0/1779605910 wait complete. 2024-09-11T00:33:03.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T00:33:03.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 -- 172.21.15.103:0/2389423904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 msgr2=0x7fd374075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.304 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 --2- 172.21.15.103:0/2389423904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 0x7fd374075580 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd364009900 tx=0x7fd36402a410 comp rx=0 tx=0).stop 2024-09-11T00:33:03.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 -- 172.21.15.103:0/2389423904 shutdown_connections 2024-09-11T00:33:03.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 --2- 172.21.15.103:0/2389423904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd37410a910 0x7fd37410cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 --2- 172.21.15.103:0/2389423904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd374075ac0 0x7fd374075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 --2- 172.21.15.103:0/2389423904 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 0x7fd374075580 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 -- 172.21.15.103:0/2389423904 >> 172.21.15.103:0/2389423904 conn(0x7fd3740fe270 msgr2=0x7fd374100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:03.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 -- 172.21.15.103:0/2389423904 shutdown_connections 2024-09-11T00:33:03.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.302+0000 7fd37ae00640 1 -- 172.21.15.103:0/2389423904 wait complete. 2024-09-11T00:33:03.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 Processor -- start 2024-09-11T00:33:03.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 -- start start 2024-09-11T00:33:03.307 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374075ac0 0x7fd37419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 0x7fd37419b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37410a910 0x7fd37419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd374110850 con 0x7fd374076f30 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741106d0 con 0x7fd37410a910 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.303+0000 7fd37ae00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd3741109d0 con 0x7fd374075ac0 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374075ac0 0x7fd37419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd36be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 0x7fd37419b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd378e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37410a910 0x7fd37419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374075ac0 0x7fd37419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.103:48846/0 (socket says 172.21.15.103:48846) 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 -- 172.21.15.103:0/3239166223 learned_addr learned my addr 172.21.15.103:0/3239166223 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:03.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 -- 172.21.15.103:0/3239166223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37410a910 msgr2=0x7fd37419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37410a910 0x7fd37419c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 -- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 msgr2=0x7fd37419b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 0x7fd37419b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 -- 172.21.15.103:0/3239166223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37419ca80 con 0x7fd374075ac0 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd378e00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37410a910 0x7fd37419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd36be00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 0x7fd37419b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd373e00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374075ac0 0x7fd37419af40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd364005ab0 tx=0x7fd364003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.304+0000 7fd371600640 1 -- 172.21.15.103:0/3239166223 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36403f070 con 0x7fd374075ac0 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.305+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37419cd10 con 0x7fd374075ac0 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.305+0000 7fd371600640 1 -- 172.21.15.103:0/3239166223 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd364031c50 con 0x7fd374075ac0 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.305+0000 7fd371600640 1 -- 172.21.15.103:0/3239166223 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364035030 con 0x7fd374075ac0 2024-09-11T00:33:03.309 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.305+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741a3440 con 0x7fd374075ac0 2024-09-11T00:33:03.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.306+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd374109720 con 0x7fd374075ac0 2024-09-11T00:33:03.310 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.308+0000 7fd371600640 1 -- 172.21.15.103:0/3239166223 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd3640317a0 con 0x7fd374075ac0 2024-09-11T00:33:03.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.309+0000 7fd371600640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd3580b5360 0x7fd3580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:03.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.310+0000 7fd371600640 1 -- 172.21.15.103:0/3239166223 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd364107060 con 0x7fd374075ac0 2024-09-11T00:33:03.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.310+0000 7fd36be00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd3580b5360 0x7fd3580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:03.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.310+0000 7fd36be00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd3580b5360 0x7fd3580b7820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009fc0 tx=0x7fd35c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:03.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.311+0000 7fd371600640 1 -- 172.21.15.103:0/3239166223 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd3640cd4e0 con 0x7fd374075ac0 2024-09-11T00:33:03.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:03 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4240189071' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:33:03.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:03 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/812484018' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:33:03.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:03 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1779605910' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-11T00:33:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:03 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4240189071' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:33:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:03 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/812484018' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:33:03.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:03 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1779605910' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-11T00:33:03.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.548+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd374063d50 con 0x7fd374075ac0 2024-09-11T00:33:03.558 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.556+0000 7fd371600640 1 -- 172.21.15.103:0/3239166223 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+827951 (secure 0 0 0) 0x7fd3640d1f10 con 0x7fd374075ac0 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: "epoch": 18, 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: "active_gid": 14150, 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: "active_name": "x", 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addrs": { 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-11T00:33:03.583 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6800", 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3157485146 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:6801", 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3157485146 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.584 INFO:teuthology.orchestra.run.smithi103.stdout: "active_addr": "172.21.15.103:6801/3157485146", 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "active_change": "2024-09-11T00:12:26.257459+0000", 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "active_mgr_features": 4540719139924082687, 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "available": true, 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "standbys": [ 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24181, 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "z", 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540719139924082687, 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.586 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.590 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.591 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.592 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.593 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.595 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-11T00:33:03.597 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.599 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-11T00:33:03.600 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.601 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.602 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.603 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-11T00:33:03.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-11T00:33:03.605 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.606 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.609 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.610 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.612 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.614 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.615 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-11T00:33:03.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-11T00:33:03.618 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.622 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.623 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.624 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-11T00:33:03.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.628 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.629 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.631 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.632 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.633 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.634 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T00:33:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T00:33:03.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T00:33:03.637 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.638 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.639 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T00:33:03.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.641 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.644 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nginx": { 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nginx", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.645 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nginx container image", 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.646 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-11T00:33:03.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-11T00:33:03.648 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_oauth2_proxy": { 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.649 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "oauth2-proxy container image", 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-11T00:33:03.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.653 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba_metrics": { 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba_metrics", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-11T00:33:03.655 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.656 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:03.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-11T00:33:03.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.659 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.660 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.661 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.662 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:03.663 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.665 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.666 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.667 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.668 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.669 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.670 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T00:33:03.671 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.672 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.673 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.674 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.675 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.676 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-11T00:33:03.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.678 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.680 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.681 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.682 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.683 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.684 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.686 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.688 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.689 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-11T00:33:03.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-11T00:33:03.689 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.689 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.689 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.689 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.690 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.691 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.692 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.693 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.694 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.695 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.697 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.698 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.699 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.700 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.701 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.702 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T00:33:03.703 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.705 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.706 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.707 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.709 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.710 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.712 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.713 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.714 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.715 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.716 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.717 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T00:33:03.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.721 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.722 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.723 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.724 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.725 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.726 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.727 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.728 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.729 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "[]", 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "{}", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.731 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.732 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.736 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T00:33:03.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.738 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.739 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.740 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.741 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.744 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.745 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.746 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.747 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.748 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T00:33:03.749 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T00:33:03.750 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.751 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.753 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.754 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.755 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.756 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.757 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.758 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.759 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.760 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.761 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.762 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.763 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.764 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.765 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.766 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.766 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.766 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.766 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.766 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.766 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.766 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.767 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.768 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T00:33:03.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:03.770 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.771 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.772 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.773 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.774 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.775 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-11T00:33:03.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.777 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.778 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.781 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.781 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.781 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.781 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.781 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.784 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.785 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.786 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.787 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.788 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.789 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T00:33:03.790 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.791 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.792 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.793 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.795 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.796 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.797 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.798 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.799 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.800 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.801 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.802 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.803 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.804 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.805 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:03.806 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.807 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.808 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.809 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-11T00:33:03.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.812 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.813 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.814 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.815 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.816 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-11T00:33:03.819 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-11T00:33:03.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.821 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.822 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.824 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.825 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:33:03.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.827 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.828 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.829 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.830 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.831 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.832 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.833 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.834 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.838 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.840 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-11T00:33:03.841 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.842 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.844 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.845 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.846 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.847 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.850 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.851 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.852 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.853 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.854 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.856 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.857 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T00:33:03.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-11T00:33:03.859 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.860 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.861 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-11T00:33:03.862 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.863 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.864 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.866 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.868 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.870 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.871 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.872 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.873 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.874 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.875 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.876 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.877 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.878 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.879 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.880 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.881 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.882 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-11T00:33:03.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.885 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.886 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.887 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.888 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.889 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.890 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.891 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.892 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.893 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.894 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.895 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.896 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.897 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.898 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.899 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.900 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.901 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.902 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.903 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.904 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.905 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.906 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.909 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.910 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:03.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.912 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-11T00:33:03.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.914 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.916 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.917 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.918 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.920 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.922 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:03.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.924 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.925 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.926 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-11T00:33:03.927 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.928 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.929 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.931 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.932 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.933 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.934 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.935 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:03.936 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:03.937 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:03.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.940 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.942 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.943 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.944 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.945 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.946 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.947 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.948 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.949 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.951 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.952 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-11T00:33:03.953 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.955 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.956 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.957 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.958 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.959 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.960 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "max_requests": { 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_requests", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.961 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:03.962 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.964 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.965 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.966 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.968 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.969 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T00:33:03.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T00:33:03.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.972 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.974 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.975 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.976 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.977 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-11T00:33:03.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:03.980 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.981 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.982 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.984 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:03.985 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.986 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.988 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.988 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.988 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.988 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:03.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:03.988 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.988 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.990 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.992 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:03.993 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:03.994 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-11T00:33:03.995 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.998 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:03.999 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.000 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.001 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smb", 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "internal_store_backend": { 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "internal_store_backend", 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.003 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.005 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.006 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.007 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.008 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.009 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.010 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "update_orchestration": { 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "update_orchestration", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.011 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.012 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.013 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.014 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.015 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.016 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.017 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.018 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.021 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.022 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.023 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.024 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.025 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.027 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.028 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.029 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.030 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.031 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.032 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.035 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.036 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.037 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.038 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.040 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.041 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.042 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-11T00:33:04.043 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-11T00:33:04.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.045 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.047 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T00:33:04.048 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.050 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.052 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-11T00:33:04.053 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.054 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.055 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.056 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.057 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.058 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.059 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.060 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.061 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.062 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.063 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.064 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.066 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.067 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.068 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.070 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.071 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.071 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.071 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.071 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.071 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.071 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.071 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.072 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.073 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.074 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.075 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.078 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T00:33:04.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.087 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.091 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.092 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.093 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.095 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.096 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "gid": 24194, 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "y", 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "mgr_features": 4540719139924082687, 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.098 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.100 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.101 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.102 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.103 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.104 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.105 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.106 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.107 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-11T00:33:04.108 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.109 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-11T00:33:04.110 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.111 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.112 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.113 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.114 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.115 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.116 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-11T00:33:04.117 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.118 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.119 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.120 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-11T00:33:04.121 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.122 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.123 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.124 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.125 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.126 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.127 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.128 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-11T00:33:04.129 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-11T00:33:04.130 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.131 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T00:33:04.132 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.133 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.134 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T00:33:04.135 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-11T00:33:04.136 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.137 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.138 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.139 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.140 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-11T00:33:04.141 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.142 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-11T00:33:04.143 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.144 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T00:33:04.145 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T00:33:04.146 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.147 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.148 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.149 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-11T00:33:04.150 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.151 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-11T00:33:04.152 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.153 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-11T00:33:04.154 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.155 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.156 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T00:33:04.157 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nginx": { 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nginx", 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.158 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nginx container image", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.159 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.160 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_oauth2_proxy": { 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T00:33:04.161 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "oauth2-proxy container image", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-11T00:33:04.162 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-11T00:33:04.163 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-11T00:33:04.164 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba_metrics": { 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba_metrics", 2024-09-11T00:33:04.165 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-11T00:33:04.166 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.167 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-11T00:33:04.168 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-11T00:33:04.169 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-11T00:33:04.170 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.171 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.172 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.175 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.176 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.177 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.178 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.179 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.180 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.181 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.181 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.181 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.181 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.181 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-11T00:33:04.181 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-11T00:33:04.181 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.182 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.183 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.184 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.185 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.186 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.187 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.188 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.189 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.190 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.192 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.192 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T00:33:04.192 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.192 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.192 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.192 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.194 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.195 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.197 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.198 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.199 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.200 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.201 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.202 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.203 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.204 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.205 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.206 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.207 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T00:33:04.208 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T00:33:04.209 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.210 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.211 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.212 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.213 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.214 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-11T00:33:04.215 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.216 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.217 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.218 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.219 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.220 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.221 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.222 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.223 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.224 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.225 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.226 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.227 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.228 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.229 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.230 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.231 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.232 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.233 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.234 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.235 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.236 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.237 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.238 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.239 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.240 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "[]", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.241 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "{}", 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.242 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.243 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.244 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.245 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.246 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.247 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.248 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.249 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.250 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.251 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.252 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.253 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.254 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T00:33:04.255 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.256 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.257 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T00:33:04.258 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.259 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T00:33:04.260 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.261 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T00:33:04.262 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T00:33:04.263 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-11T00:33:04.264 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-11T00:33:04.265 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.266 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.267 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.268 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.269 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.270 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.271 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.272 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.273 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.274 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.275 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.276 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.277 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:04.278 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.279 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-11T00:33:04.280 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-11T00:33:04.281 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.282 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.283 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.284 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.285 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.286 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.287 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.288 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.289 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.290 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.291 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-11T00:33:04.292 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-11T00:33:04.293 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.294 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.295 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.296 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.297 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.298 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.299 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.300 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.301 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.302 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.303 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.304 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.305 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.306 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.307 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.308 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.309 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-11T00:33:04.310 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.311 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.312 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.313 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.314 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.315 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.316 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.317 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.318 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.319 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-11T00:33:04.320 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.321 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.322 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.323 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-11T00:33:04.324 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.325 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.326 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.327 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.328 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.329 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.330 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.331 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.332 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.333 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.334 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.335 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.336 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.337 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.338 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.339 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.340 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.341 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.342 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.343 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.344 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.345 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.346 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.347 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.348 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.349 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.350 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.352 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.353 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.354 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.355 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.356 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.357 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.358 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.359 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-11T00:33:04.360 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.361 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.362 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.363 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.364 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.365 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.366 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.367 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.368 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.369 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.370 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.371 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.372 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.373 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.374 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.375 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.376 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.377 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.378 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.379 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.380 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.381 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.382 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.383 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.384 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.385 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.386 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.387 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.388 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.389 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.390 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.391 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.392 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.393 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.396 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.396 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.396 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.396 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.396 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-11T00:33:04.396 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.397 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.398 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.399 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.400 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.401 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.402 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.403 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.404 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.405 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.406 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.407 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.408 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.409 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.410 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.411 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.412 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.413 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.414 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.415 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.416 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-11T00:33:04.417 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.418 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.419 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.420 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.421 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.422 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.423 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.424 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.425 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.426 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.427 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.428 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.429 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.430 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T00:33:04.431 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.432 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.433 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.434 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.435 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.436 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.437 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.438 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.439 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.440 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.441 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.442 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.443 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-11T00:33:04.444 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-11T00:33:04.445 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.446 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.447 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.448 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.449 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.450 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-11T00:33:04.451 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.452 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.453 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-11T00:33:04.454 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.455 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.456 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.457 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.458 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.459 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.460 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.461 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.462 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.463 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "max_requests": { 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_requests", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T00:33:04.464 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.465 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.466 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.467 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.468 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.469 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.470 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.471 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.472 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.473 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.474 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.475 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.476 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.477 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.478 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.479 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.480 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.481 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.482 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.483 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.484 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.485 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.486 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.487 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.488 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.489 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.490 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.491 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.492 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.493 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.494 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.495 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.496 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.497 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-11T00:33:04.498 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.499 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.500 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.502 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smb", 2024-09-11T00:33:04.503 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "internal_store_backend": { 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "internal_store_backend", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T00:33:04.504 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.505 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.506 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.507 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.508 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.509 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.510 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.512 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.512 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.512 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.512 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.512 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.512 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "update_orchestration": { 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "update_orchestration", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.513 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.514 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.515 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.516 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.517 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.518 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.519 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.520 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.521 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.522 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.523 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.524 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.525 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.526 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.527 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.528 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.529 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.530 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.531 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.532 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.533 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.534 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.535 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-11T00:33:04.536 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.537 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.538 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.539 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.540 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.541 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.542 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.543 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.544 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.545 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.546 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.547 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.548 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.549 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.550 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.551 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.552 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.553 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-11T00:33:04.555 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.556 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.557 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.558 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.559 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.560 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.561 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.562 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.563 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-11T00:33:04.564 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.565 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-11T00:33:04.566 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.567 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.568 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.569 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.570 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.571 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.572 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.573 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.574 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.575 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.576 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.577 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.578 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.579 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.580 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.581 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.582 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T00:33:04.583 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.584 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.585 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.586 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.587 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.589 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.590 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.591 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.592 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.593 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.594 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.595 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.596 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.597 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.598 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "modules": [ 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard", 2024-09-11T00:33:04.599 INFO:teuthology.orchestra.run.smithi103.stdout: "iostat", 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "nfs", 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "restful" 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "available_modules": [ 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "alerts", 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.600 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.601 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.602 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.603 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.604 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.605 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.606 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_destination": { 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_destination", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.607 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_from_name": { 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_from_name", 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.608 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "Ceph", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Email From: name", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_host": { 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_host", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.609 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP server", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_password": { 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_password", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.610 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Password to authenticate with", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_port": { 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_port", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.611 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "465", 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP port", 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_sender": { 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_sender", 2024-09-11T00:33:04.612 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SMTP envelope sender", 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.613 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_ssl": { 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_ssl", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.614 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "smtp_user": { 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smtp_user", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "User to authenticate as", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.615 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.616 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "balancer", 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "active": { 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "active", 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.617 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_time": { 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_time", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.618 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0000", 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "begin_weekday": { 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "begin_weekday", 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.619 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_max_iterations": { 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_max_iterations", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.620 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "25", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "250", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_metrics": { 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_metrics", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.621 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "crush_compat_step": { 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crush_compat_step", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.622 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.5", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0.001", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "0.999", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "aggressiveness of optimization", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "end_time": { 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_time", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.623 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2359", 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "end_weekday": { 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "end_weekday", 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.624 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "0", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "6", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.625 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.626 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.627 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.628 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.629 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.630 INFO:teuthology.orchestra.run.smithi103.stdout: "min_score": { 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_score", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.631 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "upmap", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "crush-compat", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "none", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "read", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap", 2024-09-11T00:33:04.632 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap-read" 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Balancer mode", 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_ids": { 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_ids", 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.633 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.634 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.635 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_deviation": { 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_deviation", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.636 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "upmap_max_optimizations": { 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "upmap_max_optimizations", 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.637 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm", 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_down_multiplier": { 2024-09-11T00:33:04.638 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_down_multiplier", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_refresh_rate": { 2024-09-11T00:33:04.639 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_refresh_rate", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "20", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.640 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "agent_starting_port": { 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "agent_starting_port", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4721", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.641 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_ptrace": { 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_ptrace", 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.642 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_interval": { 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_interval", 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.643 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "autotune_memory_target_ratio": { 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.644 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0.7", 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm_log_destination": { 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cephadm_log_destination", 2024-09-11T00:33:04.645 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "file", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "file,syslog", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "syslog" 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.646 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "cgroups_split": { 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cgroups_split", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.647 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "config_checks_enabled": { 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_checks_enabled", 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.648 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "config_dashboard": { 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_dashboard", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.649 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_alertmanager": { 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_alertmanager", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.650 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_base": { 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_base", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.651 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Container image name, without the tag", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_elasticsearch": { 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_elasticsearch", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.652 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "elasticsearch container image", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_grafana": { 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_grafana", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.653 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_haproxy": { 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_haproxy", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "HAproxy container image", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.654 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_agent": { 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_agent", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger agent container image", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.655 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_collector": { 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_collector", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger collector container image", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.656 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_jaeger_query": { 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_jaeger_query", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Jaeger query container image", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.657 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_keepalived": { 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_keepalived", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Keepalived container image", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.658 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_loki": { 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_loki", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Loki container image", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.659 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nginx": { 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nginx", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nginx container image", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.660 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_node_exporter": { 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_node_exporter", 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.661 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_nvmeof": { 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_nvmeof", 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.662 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Nvme-of container image", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_oauth2_proxy": { 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_oauth2_proxy", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.663 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "oauth2-proxy container image", 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_prometheus": { 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_prometheus", 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.664 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Prometheus container image", 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_promtail": { 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_promtail", 2024-09-11T00:33:04.665 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Promtail container image", 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba": { 2024-09-11T00:33:04.666 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB container image", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_samba_metrics": { 2024-09-11T00:33:04.667 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_samba_metrics", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.668 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "container_image_snmp_gateway": { 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_image_snmp_gateway", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "SNMP Gateway container image", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.669 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "container_init": { 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "container_init", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.670 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "daemon_cache_timeout": { 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "daemon_cache_timeout", 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.671 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "default_cephadm_command_timeout": { 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "900", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.672 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_registry": { 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "default_registry", 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "docker.io", 2024-09-11T00:33:04.673 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "device_cache_timeout": { 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_cache_timeout", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1800", 2024-09-11T00:33:04.674 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache device inventory", 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "device_enhanced_scan": { 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_enhanced_scan", 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.675 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "facts_cache_timeout": { 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "facts_cache_timeout", 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.676 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "seconds to cache host facts data", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "grafana_dashboards_path": { 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "grafana_dashboards_path", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.677 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "host_check_interval": { 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "host_check_interval", 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.678 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to perform a host check", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "hw_monitoring": { 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hw_monitoring", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.679 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "inventory_list_all": { 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "inventory_list_all", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.680 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.681 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.682 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "log_refresh_metadata": { 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_refresh_metadata", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T00:33:04.683 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.684 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.685 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.686 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.688 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "*", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.689 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max_count_per_host": { 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_count_per_host", 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.690 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of daemons per service per host", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "max_osd_draining_count": { 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_osd_draining_count", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T00:33:04.691 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "migration_current": { 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "migration_current", 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.692 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "internal - do not modify", 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "mode": { 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mode", 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "root", 2024-09-11T00:33:04.693 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm-package", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "root" 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "oob_default_addr": { 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "oob_default_addr", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.694 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "169.254.1.1", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "prometheus_alerts_path": { 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus_alerts_path", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.695 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_insecure": { 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_insecure", 2024-09-11T00:33:04.696 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_password": { 2024-09-11T00:33:04.697 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_password", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_url": { 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_url", 2024-09-11T00:33:04.698 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.699 INFO:teuthology.orchestra.run.smithi103.stdout: "registry_username": { 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "registry_username", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.700 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "secure_monitoring_stack": { 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secure_monitoring_stack", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.701 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "service_discovery_port": { 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "service_discovery_port", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8765", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "cephadm service discovery port", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.702 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.703 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "ssh_config_file": { 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssh_config_file", 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T00:33:04.704 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "use_agent": { 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_agent", 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.705 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "use_repo_digest": { 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "use_repo_digest", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.706 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_failed_host_check": { 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_failed_host_check", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.707 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_daemons": { 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_daemons", 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.708 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.709 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.709 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.709 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T00:33:04.709 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.709 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.709 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.710 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.710 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_on_stray_hosts": { 2024-09-11T00:33:04.710 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_on_stray_hosts", 2024-09-11T00:33:04.710 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.710 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.710 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.710 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crash", 2024-09-11T00:33:04.711 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.712 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.713 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.714 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.715 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.716 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.717 INFO:teuthology.orchestra.run.smithi103.stdout: "retain_interval": { 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retain_interval", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "31536000", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.718 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_recent_interval": { 2024-09-11T00:33:04.719 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_recent_interval", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1209600", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.720 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dashboard", 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.721 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.722 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.723 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_ENABLED": { 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.724 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.725 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.726 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.727 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.728 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.729 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.730 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.731 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.732 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.733 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.734 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.735 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.736 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_URL": { 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_URL", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.737 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.739 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.740 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.741 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.742 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "[]", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "{}", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.744 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T00:33:04.745 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.746 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.747 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.748 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.749 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.750 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.751 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.752 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.754 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.755 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.756 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.757 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "45", 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.758 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.759 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "admin", 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.760 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.761 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.762 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.763 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.764 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.765 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "cross_origin_url": { 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cross_origin_url", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.766 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "crt_file": { 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "crt_file", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.767 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "debug": { 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "debug", 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.768 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Enable/disable debug options", 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "jwt_token_ttl": { 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "jwt_token_ttl", 2024-09-11T00:33:04.769 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "28800", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.770 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.771 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.772 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.773 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.774 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.775 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "motd": { 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "motd", 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.776 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "The message of the day", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect_resolve_ip_addr": { 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.777 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.778 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.779 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8080", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.780 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.781 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl_server_port": { 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl_server_port", 2024-09-11T00:33:04.782 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8443", 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-11T00:33:04.783 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "redirect", 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.784 INFO:teuthology.orchestra.run.smithi103.stdout: "redirect" 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.785 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "url_prefix": { 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url_prefix", 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.786 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.787 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_monitoring": { 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_monitoring", 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.788 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "monitor device health metrics", 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.789 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.790 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.791 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.792 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.793 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "mark_out_threshold": { 2024-09-11T00:33:04.794 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mark_out_threshold", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "2419200", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.795 INFO:teuthology.orchestra.run.smithi103.stdout: "pool_name": { 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pool_name", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "device_health_metrics", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.796 INFO:teuthology.orchestra.run.smithi103.stdout: "retention_period": { 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "retention_period", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15552000", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long to retain device health metrics", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_frequency": { 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_frequency", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.798 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "self_heal": { 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "self_heal", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.799 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.800 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.801 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.802 INFO:teuthology.orchestra.run.smithi103.stdout: "warn_threshold": { 2024-09-11T00:33:04.802 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "warn_threshold", 2024-09-11T00:33:04.802 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.804 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.804 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.804 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7257600", 2024-09-11T00:33:04.804 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.804 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.804 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.804 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "diskprediction_local", 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.805 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.806 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.807 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.808 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.809 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.810 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "predict_interval": { 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predict_interval", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.811 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "86400", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "predictor_model": { 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "predictor_model", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.812 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "prophetstor", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.813 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "600", 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.814 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.815 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "influx", 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": false, 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "influxdb python module not found", 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "batch_size": { 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "batch_size", 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.816 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5000", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "database": { 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "database", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.817 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "ceph", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": { 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "hostname", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.818 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server hostname", 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.819 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "30", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "5", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.820 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.821 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.822 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.823 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.824 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "password": { 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "password", 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.826 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "password of InfluxDB server user", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "port": { 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "port", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "8086", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.827 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "InfluxDB server port", 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.828 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "ssl": { 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ssl", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "false", 2024-09-11T00:33:04.829 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "threads": { 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threads", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.830 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "32", 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "username": { 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "username", 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.831 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "username of InfluxDB server user", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "verify_ssl": { 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "verify_ssl", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.832 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "true", 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "insights", 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.833 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.834 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.835 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.836 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.837 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.838 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.839 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "iostat", 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.840 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.841 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.842 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.843 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.844 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.845 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.846 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "k8sevents", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_event_retention_days": { 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "ceph_event_retention_days", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "7", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T00:33:04.847 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "config_check_secs": { 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "config_check_secs", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "10", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T00:33:04.848 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.849 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.850 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.851 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.852 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.853 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.854 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "localpool", 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.855 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "failure_domain": { 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "failure_domain", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "host", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "failure domain for any created local pool", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.856 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.857 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.858 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.859 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.860 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.861 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "min_size": { 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "min_size", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.862 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default min_size for any created local pool", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "num_rep": { 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "num_rep", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.863 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default replica count for any created local pool", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_num": { 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_num", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.864 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "128", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "prefix": { 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prefix", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.865 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "name prefix for any created local pool", 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.866 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "subtree": { 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "subtree", 2024-09-11T00:33:04.867 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "rack", 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mds_autoscaler", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.869 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.870 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.871 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.872 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.873 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirroring", 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.875 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.876 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.877 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.878 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.879 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.880 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "nfs", 2024-09-11T00:33:04.881 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.882 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.883 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.884 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.885 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.886 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.887 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.888 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "fail_fs": { 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "fail_fs", 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.889 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.890 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.891 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.892 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.893 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.894 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.895 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator": { 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "orchestrator", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "cephadm", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "rook", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "test_orchestrator" 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Orchestrator backend", 2024-09-11T00:33:04.896 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.897 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_perf_query", 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.898 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.899 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.900 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.901 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.902 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.903 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.904 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "osd_support", 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.905 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.906 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.907 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.908 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.909 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.910 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.911 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "pg_autoscaler", 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.912 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.913 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.914 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.915 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.916 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.917 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.918 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.919 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "threshold": { 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "threshold", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "3.0", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1.0", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.920 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "scaling threshold", 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "progress", 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.921 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_pg_recovery_event": { 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_pg_recovery_event", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-11T00:33:04.922 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.923 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.924 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.925 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.926 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.927 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.928 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "max_completed_events": { 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_completed_events", 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "50", 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "number of past completed events to remember", 2024-09-11T00:33:04.929 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "sleep_interval": { 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sleep_interval", 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.930 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "how long the module is going to sleep", 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.931 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.932 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "prometheus", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "cache": { 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "cache", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.933 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "exclude_perf_counters": { 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "exclude_perf_counters", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T00:33:04.934 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.935 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.936 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.937 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.938 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools": { 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.939 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.940 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "300", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "scrape_interval": { 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "scrape_interval", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.941 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15.0", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "::", 2024-09-11T00:33:04.942 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "9283", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.943 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.944 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "stale_cache_strategy": { 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stale_cache_strategy", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "log", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.945 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_behaviour": { 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_behaviour", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "default", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "default", 2024-09-11T00:33:04.946 INFO:teuthology.orchestra.run.smithi103.stdout: "error" 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "standby_error_status_code": { 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "standby_error_status_code", 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "400", 2024-09-11T00:33:04.947 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "599", 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.948 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.949 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.950 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.951 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.952 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_snap_create": { 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_snap_create", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.953 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "mirror_snapshot_schedule": { 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.954 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.955 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.956 INFO:teuthology.orchestra.run.smithi103.stdout: "trash_purge_schedule": { 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "trash_purge_schedule", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.957 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "restful", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enable_auth": { 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enable_auth", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.958 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "key_file": { 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "key_file", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.959 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.960 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.961 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.962 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.963 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.964 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "max_requests": { 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_requests", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "500", 2024-09-11T00:33:04.965 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "server_addr": { 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_addr", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.966 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "server_port": { 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "server_port", 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.967 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.968 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rgw", 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.969 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.970 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.971 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.972 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.973 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T00:33:04.974 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "5", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.975 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rook", 2024-09-11T00:33:04.976 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.977 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.978 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.979 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.980 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.981 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.982 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "storage_class": { 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "storage_class", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "local", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.983 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:04.984 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "selftest", 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.985 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:04.986 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.987 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:04.988 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.989 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "roption1": { 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption1", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.990 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "roption2": { 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "roption2", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "xyz", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.991 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption1": { 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption1", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.992 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption2": { 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption2", 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.993 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption3": { 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption3", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "float", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.994 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption4": { 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption4", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.995 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption5": { 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption5", 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.996 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption6": { 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption6", 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.997 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "rwoption7": { 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rwoption7", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:04.998 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "1", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "42", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:04.999 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "testkey": { 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testkey", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.000 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "testlkey": { 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testlkey", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.001 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "testnewline": { 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "testnewline", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.002 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "smb", 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.003 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "internal_store_backend": { 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "internal_store_backend", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.004 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.005 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.006 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.007 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.008 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.009 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.010 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "update_orchestration": { 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "update_orchestration", 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.011 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.012 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snap_schedule", 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "allow_m_granularity": { 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "allow_m_granularity", 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.013 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "dump_on_update": { 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "dump_on_update", 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.014 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "dump database to debug log on update", 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.015 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.016 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.017 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.018 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.019 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.020 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.021 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "stats", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.022 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.023 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.024 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.025 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.026 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.027 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "status", 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.028 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.029 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.030 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.031 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.032 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.033 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telegraf", 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "address": { 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "address", 2024-09-11T00:33:05.034 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:05.035 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "15", 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.036 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.037 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.038 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.039 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.040 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.041 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.042 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "telemetry", 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_basic": { 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_basic", 2024-09-11T00:33:05.043 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_crash": { 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_crash", 2024-09-11T00:33:05.044 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_device": { 2024-09-11T00:33:05.045 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_device", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_ident": { 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_ident", 2024-09-11T00:33:05.046 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "channel_perf": { 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "channel_perf", 2024-09-11T00:33:05.047 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "contact": { 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "contact", 2024-09-11T00:33:05.048 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "description": { 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "description", 2024-09-11T00:33:05.049 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "device_url": { 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "device_url", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.050 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "enabled": { 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "enabled", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.051 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.052 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "24", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "8", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "last_opt_revision": { 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "last_opt_revision", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "1", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard": { 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.054 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "leaderboard_description": { 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "leaderboard_description", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.055 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.056 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.057 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.058 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.059 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.060 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "organization": { 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "organization", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.061 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "proxy": { 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "proxy", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.062 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.063 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.064 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "url": { 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "url", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.065 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "test_orchestrator", 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.066 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.067 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.068 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.069 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.070 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.071 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.072 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.073 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.074 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.075 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.076 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.077 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.078 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "max_concurrent_clones": { 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "max_concurrent_clones", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.079 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "4", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "periodic_async_work": { 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "periodic_async_work", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Periodically check for async work", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_delay": { 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_delay", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.081 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "snapshot_clone_no_wait": { 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.082 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "True", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.083 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix", 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "can_run": true, 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "error_string": "", 2024-09-11T00:33:05.084 INFO:teuthology.orchestra.run.smithi103.stdout: "module_options": { 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "discovery_interval": { 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "discovery_interval", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "uint", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "100", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "identifier": { 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "identifier", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.086 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "interval": { 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "interval", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "secs", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "60", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.087 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "log_level": { 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_level", 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.088 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster": { 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster", 2024-09-11T00:33:05.089 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_cluster_level": { 2024-09-11T00:33:05.090 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_cluster_level", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "info", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [ 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "critical", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "debug", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "error", 2024-09-11T00:33:05.091 INFO:teuthology.orchestra.run.smithi103.stdout: "info", 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "warning" 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "log_to_file": { 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "log_to_file", 2024-09-11T00:33:05.092 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "bool", 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "False", 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "sqlite3_killpoint": { 2024-09-11T00:33:05.093 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "sqlite3_killpoint", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "dev", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 1, 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "0", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.094 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_host": { 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_host", 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "", 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.095 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_port": { 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_port", 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "int", 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "10051", 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.096 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "zabbix_sender": { 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "zabbix_sender", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "str", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "level": "advanced", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "flags": 0, 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "min": "", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": "", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "enum_allowed": [], 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "desc": "", 2024-09-11T00:33:05.097 INFO:teuthology.orchestra.run.smithi103.stdout: "long_desc": "", 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "tags": [], 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "see_also": [] 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "services": { 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "dashboard": "https://172.21.15.103:8443/" 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "always_on_modules": { 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "octopus": [ 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-11T00:33:05.098 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "pacific": [ 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-11T00:33:05.099 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "quincy": [ 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-11T00:33:05.100 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "reef": [ 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-11T00:33:05.101 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: ], 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "squid": [ 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "balancer", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "crash", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "devicehealth", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "orchestrator", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "pg_autoscaler", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "progress", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "rbd_support", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "status", 2024-09-11T00:33:05.102 INFO:teuthology.orchestra.run.smithi103.stdout: "telemetry", 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "volumes" 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "force_disabled_modules": {}, 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "last_failure_osd_epoch": 3, 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "active_clients": [ 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "devicehealth", 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 2830913090 2024-09-11T00:33:05.103 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "libcephsqlite", 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 4033999187 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:05.104 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "rbd_support", 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 460259377 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.105 INFO:teuthology.orchestra.run.smithi103.stdout: "name": "volumes", 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: "addrvec": [ 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: { 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v2", 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "172.21.15.103:0", 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 3484002684 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:05.106 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-09-11T00:33:05.107 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:33:05.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd3580b5360 msgr2=0x7fd3580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:05.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd3580b5360 0x7fd3580b7820 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009fc0 tx=0x7fd35c005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:05.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374075ac0 msgr2=0x7fd37419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:05.108 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374075ac0 0x7fd37419af40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd364005ab0 tx=0x7fd364003b90 comp rx=0 tx=0).stop 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 shutdown_connections 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd3580b5360 0x7fd3580b7820 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd37410a910 0x7fd37419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd374076f30 0x7fd37419b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 --2- 172.21.15.103:0/3239166223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd374075ac0 0x7fd37419af40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 >> 172.21.15.103:0/3239166223 conn(0x7fd3740fe270 msgr2=0x7fd3741068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 shutdown_connections 2024-09-11T00:33:05.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:03.564+0000 7fd37ae00640 1 -- 172.21.15.103:0/3239166223 wait complete. 2024-09-11T00:33:05.117 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T00:33:05.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T00:33:05.117 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:04 smithi103 ceph-mon[30578]: pgmap v700: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 849 KiB/s wr, 456 op/s 2024-09-11T00:33:05.117 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:04 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3239166223' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T00:33:05.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:04 smithi183 ceph-mon[35934]: pgmap v700: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 849 KiB/s wr, 456 op/s 2024-09-11T00:33:05.118 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:04 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3239166223' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T00:33:05.119 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:03 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4240189071' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-11T00:33:05.119 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:03 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/812484018' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-11T00:33:05.119 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:03 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1779605910' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-11T00:33:05.119 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:04 smithi117 ceph-mon[35525]: pgmap v700: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 849 KiB/s wr, 456 op/s 2024-09-11T00:33:05.119 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:04 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3239166223' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-11T00:33:05.502 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:06.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:06 smithi103 ceph-mon[30578]: pgmap v701: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 832 KiB/s wr, 366 op/s 2024-09-11T00:33:06.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:06 smithi183 ceph-mon[35934]: pgmap v701: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 832 KiB/s wr, 366 op/s 2024-09-11T00:33:06.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:06 smithi117 ceph-mon[35525]: pgmap v701: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 832 KiB/s wr, 366 op/s 2024-09-11T00:33:06.862 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:33:06.862 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:06.862 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2992, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 448, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.006245009, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000057543, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.003286074, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.006392739, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002130913 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.863 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 211, 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 208, 2024-09-11T00:33:06.864 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 522272, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 127904, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.043133458, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.024367729, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.023813957, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 207, 2024-09-11T00:33:06.865 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.014680837, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000070921 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 522272, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 127904 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 158, 2024-09-11T00:33:06.866 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 108048, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.019931124, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.016204716, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001598248, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 156, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.011082408, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000071041 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.867 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 108048 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:06.868 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:06.869 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:06.870 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 148967, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 29, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:06.871 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 968, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-11T00:33:06.872 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 212, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 492442, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 212, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 212, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 492442, 2024-09-11T00:33:06.873 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-11T00:33:06.874 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:06.874 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:06.874 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:06.874 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:06.874 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:06.874 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:33:08.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:08 smithi183 ceph-mon[35934]: pgmap v702: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 309 KiB/s wr, 241 op/s 2024-09-11T00:33:08.652 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:08 smithi103 ceph-mon[30578]: pgmap v702: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 309 KiB/s wr, 241 op/s 2024-09-11T00:33:08.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:08 smithi117 ceph-mon[35525]: pgmap v702: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 309 KiB/s wr, 241 op/s 2024-09-11T00:33:08.701 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-11T00:33:08.702 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-11T00:33:08.702 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-11T00:33:08.702 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-11T00:33:08.702 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-09-11T00:33:08.702 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T00:33:08.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T00:33:09.074 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:09.742 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:33:09.742 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:09.742 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-11T00:33:09.742 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-11T00:33:09.742 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-11T00:33:09.742 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-11T00:33:09.742 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.003074489, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000487157, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:09.743 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 164, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 109104, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.020366007, 2024-09-11T00:33:09.744 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.016401728, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001935960, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 162, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.011165575, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000068923 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:09.745 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 109104 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 222, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 217, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 527168, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 132832, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.045637959, 2024-09-11T00:33:09.746 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.024020339, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.026414298, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 217, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.019892124, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000091668 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 527168, 2024-09-11T00:33:09.747 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 132832 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:09.748 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:09.749 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:09.750 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 149007, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:09.751 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-11T00:33:09.752 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 968, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 219, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 493352, 2024-09-11T00:33:09.753 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 219, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 219, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 493352, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:09.754 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:33:10.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:10 smithi183 ceph-mon[35934]: pgmap v703: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 297 KiB/s wr, 159 op/s 2024-09-11T00:33:10.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:10 smithi103 ceph-mon[30578]: pgmap v703: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 297 KiB/s wr, 159 op/s 2024-09-11T00:33:10.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:10 smithi117 ceph-mon[35525]: pgmap v703: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 297 KiB/s wr, 159 op/s 2024-09-11T00:33:10.759 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T00:33:10.759 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T00:33:10.759 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-11T00:33:10.759 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-11T00:33:10.759 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-11T00:33:10.759 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T00:33:10.760 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T00:33:11.133 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 4, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2992, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 448, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.006689141, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000095376, 2024-09-11T00:33:11.792 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.003418864, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.006183202, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002061067 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.793 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 160, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 108400, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.021086790, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.017009077, 2024-09-11T00:33:11.794 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001920033, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 158, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.009876342, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000062508 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 108400 2024-09-11T00:33:11.795 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 214, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 211, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 523088, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 129824, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.045055725, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.025741195, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.024126928, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:11.796 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 210, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.017600589, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000083812 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 523088, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 129824 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-11T00:33:11.797 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:11.798 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:11.799 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 149998, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:11.800 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 968, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-11T00:33:11.801 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 215, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 492832, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 215, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-11T00:33:11.802 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 215, 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 492832, 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:11.803 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:33:12.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:12 smithi183 ceph-mon[35934]: pgmap v704: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 67 op/s 2024-09-11T00:33:12.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:12 smithi103 ceph-mon[30578]: pgmap v704: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 67 op/s 2024-09-11T00:33:12.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:12 smithi117 ceph-mon[35525]: pgmap v704: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 67 op/s 2024-09-11T00:33:12.766 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-11T00:33:12.766 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-11T00:33:12.766 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-11T00:33:12.766 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-11T00:33:12.766 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-09-11T00:33:12.766 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T00:33:12.767 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T00:33:13.137 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 0, 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 2, 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 0, 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 0, 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 0, 2024-09-11T00:33:14.080 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.002905999, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.000440197, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.081 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 224, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 219, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 527712, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 134112, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 2, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 1, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.048364258, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.025939398, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.026571330, 2024-09-11T00:33:14.082 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 219, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.022223597, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000101477 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 527712, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 134112 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:14.083 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 1, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 165, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 160, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 109280, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 3, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.019920790, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.016382262, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.001548974, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 163, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.012900891, 2024-09-11T00:33:14.084 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000079146 2024-09-11T00:33:14.085 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.085 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:14.085 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:14.085 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:14.085 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:14.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.086 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:14.086 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 109280 2024-09-11T00:33:14.086 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.086 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-09-11T00:33:14.086 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:14.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:14.088 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:14.089 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 149998, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 30, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 0, 2024-09-11T00:33:14.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 968, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 2, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-09-11T00:33:14.091 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 221, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 493612, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 221, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 221, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 493612, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-09-11T00:33:14.092 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-09-11T00:33:14.093 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:14.093 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-09-11T00:33:14.093 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-09-11T00:33:14.531 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:14 smithi183 ceph-mon[35934]: pgmap v705: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 17 op/s 2024-09-11T00:33:14.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:14 smithi103 ceph-mon[30578]: pgmap v705: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 17 op/s 2024-09-11T00:33:14.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:14 smithi117 ceph-mon[35525]: pgmap v705: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 17 op/s 2024-09-11T00:33:14.886 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T00:33:14.887 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T00:33:14.887 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-11T00:33:14.887 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-11T00:33:14.887 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-11T00:33:14.887 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T00:33:14.887 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T00:33:15.147 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:33:15.655 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-11T00:33:15.655 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:15.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 36614, 2024-09-11T00:33:15.655 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 37348, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 10130321, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 56761510, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 126, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 7, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 2.700866720, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 1.194724421, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.689436044, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.693731948, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 37348, 2024-09-11T00:33:15.656 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 4.935480171, 2024-09-11T00:33:15.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000132148 2024-09-11T00:33:15.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.657 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:15.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6097, 2024-09-11T00:33:15.657 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001069409, 2024-09-11T00:33:15.657 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000175 2024-09-11T00:33:15.657 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.660 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 667264, 2024-09-11T00:33:15.660 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 139056 2024-09-11T00:33:15.660 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.660 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 57120, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 106190, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 17625817, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 62384704, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 11, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 4.580977773, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 2.607231313, 2024-09-11T00:33:15.661 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 1.709818637, 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.502143521, 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 106188, 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 11.824767451, 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000111356 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:15.662 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 66319, 2024-09-11T00:33:15.663 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.018418828, 2024-09-11T00:33:15.663 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000277 2024-09-11T00:33:15.663 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.663 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 56464, 2024-09-11T00:33:15.663 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 404064 2024-09-11T00:33:15.664 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.664 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:15.664 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 22210, 2024-09-11T00:33:15.664 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 22610, 2024-09-11T00:33:15.664 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 6728218, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 27006804, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 13, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 6, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 1.710328548, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.742384889, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.462694163, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.473169708, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 22608, 2024-09-11T00:33:15.665 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 8.148363029, 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000360419 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 7626, 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.001318111, 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000172 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:33:15.666 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 90293, 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 19.819983156, 2024-09-11T00:33:15.667 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000219507 2024-09-11T00:33:15.668 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.668 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.668 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-11T00:33:15.668 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-11T00:33:15.668 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-11T00:33:15.668 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2539, 2024-09-11T00:33:15.669 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.282773990, 2024-09-11T00:33:15.669 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000111372 2024-09-11T00:33:15.669 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.669 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.669 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-11T00:33:15.669 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 18838, 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 18594, 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 18594, 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 145.973852835, 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.007850589 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-11T00:33:15.670 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 244, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 11, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 909, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 23, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 1, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 25, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 6, 2024-09-11T00:33:15.671 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 9, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 10, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 15, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 5780, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 2, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 2, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 4, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 22359, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 18609, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 161, 2024-09-11T00:33:15.672 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 59, 2024-09-11T00:33:15.673 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 2, 2024-09-11T00:33:15.673 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-11T00:33:15.673 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 2, 2024-09-11T00:33:15.673 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 637, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 91, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 5292, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 10, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 528, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 4, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 234, 2024-09-11T00:33:15.674 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 8, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 10903, 2024-09-11T00:33:15.675 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 16287, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 7863, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 14931, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 10472, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 381, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 12655 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 142, 2024-09-11T00:33:15.676 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 511, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 281, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 340, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 229, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 1351, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 1348, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 3, 2024-09-11T00:33:15.677 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 7, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 2242, 2024-09-11T00:33:15.678 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 91, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 25, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-11T00:33:15.679 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 38442, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 38430, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 38430, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 12, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 42, 2024-09-11T00:33:15.680 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 80, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 79, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 79, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 1, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 1, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 1, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 82594641, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 82616259, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 0, 2024-09-11T00:33:15.681 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 25402, 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 54.435075802, 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002142944 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 0 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 28, 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 6335, 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 6307, 2024-09-11T00:33:15.682 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 27, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 524, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 497, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 25, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 7257, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 7232, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 2, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 6325, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 6323, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 96040, 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 350089216 2024-09-11T00:33:15.683 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 37408, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 135, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 18838, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 40, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 262, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 5779, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 49.985030079, 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.008649425 2024-09-11T00:33:15.684 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 223, 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.201232177, 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.005386691 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.685 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3, 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.003121438, 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001040479 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1828, 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.939514722, 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000513957 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000188690, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000188690 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.687 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.688 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-11T00:33:15.689 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 68, 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.057506029, 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000845676 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-11T00:33:15.690 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 348, 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.073186276, 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000210305 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-11T00:33:15.691 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 96, 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.487562896, 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.005078780 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 89, 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.165568178, 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001860316 2024-09-11T00:33:15.692 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-11T00:33:15.693 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 11, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.125574369, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.011415851 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.694 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 8843, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 90.369261960, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.010219299 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 8, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.003180213, 2024-09-11T00:33:15.695 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000397526 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 8, 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.022495958, 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.002811994 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.696 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 6, 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.004573299, 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000762216 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1283, 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2.535856551, 2024-09-11T00:33:15.697 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.001976505 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.698 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 3, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 2, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 1169, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 1169, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 586, 2024-09-11T00:33:15.699 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 360, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 360, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:15.700 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:15.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:15.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:15.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:15.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:15.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:15.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:15.701 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:15.702 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 279241, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 100, 2024-09-11T00:33:15.703 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8856, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 166, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:15.704 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 118709, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 611, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.705 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 74147, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 87018473, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 37, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 74147, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 74147, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1795.208246088, 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.024211475 2024-09-11T00:33:15.706 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 74147, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 87229 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 74147, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 46, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 74101, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 655, 2024-09-11T00:33:15.707 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 7365, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 35, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 25437, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 179, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 7, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 3281, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-11T00:33:15.708 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 4, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 11417, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-11T00:33:15.709 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:33:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 36717, 2024-09-11T00:33:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-11T00:33:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-11T00:33:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-11T00:33:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-11T00:33:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-11T00:33:15.710 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 92, 2024-09-11T00:33:15.711 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 14, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 12, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 12, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 1969, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 22, 2024-09-11T00:33:15.712 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 141 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 1, 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 30, 2024-09-11T00:33:15.713 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 10, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 2709, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 1348, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 115940, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 24581135, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:15.714 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 115940, 2024-09-11T00:33:15.715 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:15.715 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:15.715 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 115940, 2024-09-11T00:33:15.715 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 24581135, 2024-09-11T00:33:15.715 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:15.715 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.715 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.716 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.716 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.716 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 74147, 2024-09-11T00:33:15.717 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 87165117, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 74147, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 87165117, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-11T00:33:15.718 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 74147, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 74147, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 74147, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 74147, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.719 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 1348, 2024-09-11T00:33:15.720 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 146820, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 1348, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 35, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 146820, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.721 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x559166f40dc0": { 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 38442, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 78421955, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 38442, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:15.722 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 25402, 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 78421955, 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 2253 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:15.723 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-11T00:33:16.508 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:16 smithi103 ceph-mon[30578]: pgmap v706: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-11T00:33:16.530 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:16 smithi183 ceph-mon[35934]: pgmap v706: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-11T00:33:16.553 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:16 smithi117 ceph-mon[35525]: pgmap v706: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-11T00:33:16.580 INFO:tasks.check_counter:Daemon mds.b mds.exported=91 2024-09-11T00:33:16.580 INFO:tasks.check_counter:Daemon mds.b mds.imported=10 2024-09-11T00:33:16.580 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=142 2024-09-11T00:33:16.580 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=511 2024-09-11T00:33:16.580 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=23 2024-09-11T00:33:16.580 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T00:33:16.581 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T00:33:16.830 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:33:17.349 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-11T00:33:17.349 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:17.349 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 254, 2024-09-11T00:33:17.349 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 229, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 637250, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 135412, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.034346963, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.016549184, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.022257338, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000016856, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 228, 2024-09-11T00:33:17.350 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.011175629, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000049015 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 636416, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 135088 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 602, 2024-09-11T00:33:17.351 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 765, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 3722602, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 543891, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.070051948, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.033239446, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.014032228, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.016979539, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 763, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.028666772, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000037571 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.352 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 352768 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.353 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1470, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1470, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 4635110, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 466694, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.154730071, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.071437065, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.035629602, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.036492235, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1469, 2024-09-11T00:33:17.354 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.057445540, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000039105 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-11T00:33:17.355 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3516, 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.089516994, 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000025459 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.356 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.357 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 1637, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 1637, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 1637, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-09-11T00:33:17.358 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-11T00:33:17.359 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:33:17.360 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-09-11T00:33:17.361 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-11T00:33:17.362 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-11T00:33:17.363 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-09-11T00:33:17.364 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 4602, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 6, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 2, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 11962702, 2024-09-11T00:33:17.365 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 11962103, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 4602 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 12, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 1669, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 1657, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 10, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 107, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 97, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 1637, 2024-09-11T00:33:17.366 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 3610, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 1973, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 53336, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 260960256 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-11T00:33:17.367 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.368 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.369 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.370 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.371 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.372 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.373 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.374 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.375 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.376 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.377 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-11T00:33:17.378 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-11T00:33:17.379 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 1, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 0, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 2093, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:17.380 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:17.381 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:17.382 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:17.383 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 198373, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 60, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:17.384 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8856, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 166, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 1154945, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 7110, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:17.385 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 2068, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 2068, 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-11T00:33:17.386 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2068, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.684965468, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000814780 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2068, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2069 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 2068, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 2068, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-11T00:33:17.387 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1297, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 769, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-11T00:33:17.388 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-11T00:33:17.389 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-11T00:33:17.390 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 92, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 7, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 6, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 6, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-11T00:33:17.391 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2322, 2024-09-11T00:33:17.392 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 8770127, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2322, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2322, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 8770127, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-11T00:33:17.393 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2068, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 7770526, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2068, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 7770526, 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:17.394 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2068, 2024-09-11T00:33:17.395 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 2068, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2068, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2068, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-11T00:33:17.396 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.397 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x55e8d9199040": { 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:17.398 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.399 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:17.399 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-11T00:33:17.399 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-11T00:33:17.399 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:17.399 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-11T00:33:18.099 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T00:33:18.099 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-11T00:33:18.099 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-11T00:33:18.099 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-11T00:33:18.099 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-11T00:33:18.099 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T00:33:18.099 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T00:33:18.346 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:33:18.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:18 smithi117 ceph-mon[35525]: pgmap v707: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T00:33:18.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:18 smithi103 ceph-mon[30578]: pgmap v707: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T00:33:18.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:18 smithi183 ceph-mon[35934]: pgmap v707: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T00:33:18.860 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-11T00:33:18.860 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:18.860 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1215, 2024-09-11T00:33:18.860 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 1215, 2024-09-11T00:33:18.860 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 21620207, 2024-09-11T00:33:18.860 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 385553, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 7, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.127687518, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.055966575, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.035893932, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.026596332, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 1214, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.048373303, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000039846 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.861 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 469, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 631, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 8507155, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 503110, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 5, 2024-09-11T00:33:18.862 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 4, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.058393541, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.026381784, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.014955399, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.012896971, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 629, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.019863619, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000031579 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.863 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 354224 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 711, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 686, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 4953482, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 280088, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 3, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.073352269, 2024-09-11T00:33:18.864 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.034833692, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.032523137, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.008557190, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 685, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.022635236, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000033044 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.865 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 635872, 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 133808 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 1, 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-MDSRank": { 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 3575, 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.081809205, 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000022883 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.866 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "finisher-PurgeQueue": { 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "queue_len": 0, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "complete_latency": { 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "mds": { 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "request": 0, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "reply": 0, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "reply_latency": { 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.867 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_reply": 0, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "forward": 0, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_complete": 0, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_fetch_keys": 0, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_commit": 0, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_split": 0, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_merge": 0, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes": 5826, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_top": 23, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_bottom": 5803, 2024-09-11T00:33:18.868 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pin_tail": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_pinned": 5792, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_expired": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "inodes_with_caps": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "caps": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "subtrees": 1, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_hit": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_forward": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_discover": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_remote_ino": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "traverse_lock": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "load_cent": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "q": 0, 2024-09-11T00:33:18.869 INFO:teuthology.orchestra.run.smithi117.stdout: "exported": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "exported_inodes": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "imported": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "imported_inodes": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_dir_fetch": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "openino_peer_discover": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rfiles": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rbytes": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "root_rsnaps": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_set_tag": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:33:18.870 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "scrub_file_inodes": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_cap_release": 0, 2024-09-11T00:33:18.871 INFO:teuthology.orchestra.run.smithi117.stdout: "process_request_cap_release": 0 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_cache": { 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_update_receipt": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_try_discover": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_send_discover": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "dir_handle_discover": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_delayed": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_created": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_enqueued": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_reintegrated": 0, 2024-09-11T00:33:18.872 INFO:teuthology.orchestra.run.smithi117.stdout: "strays_migrated": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_processing": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_started": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "recovery_completed": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_exportdir": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_flush": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_fragstats": 0, 2024-09-11T00:33:18.873 INFO:teuthology.orchestra.run.smithi117.stdout: "ireq_inodestats": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_started": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_succeeded": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "uninline_write_failed": 0 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_log": { 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "evlrg": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "evadd": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "evex": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "evtrm": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "ev": 15464, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "evexg": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "evexd": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "segadd": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "segex": 0, 2024-09-11T00:33:18.874 INFO:teuthology.orchestra.run.smithi117.stdout: "segtrm": 0, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "seg": 21, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "segmjr": 3, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "segexg": 0, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "segexd": 0, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "expos": 4194304, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "wrpos": 38024679, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "rdpos": 38024490, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "jlat": { 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "replayed": 15464 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_mem": { 2024-09-11T00:33:18.875 INFO:teuthology.orchestra.run.smithi117.stdout: "ino": 15, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "ino+": 5464, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "ino-": 5449, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "dir": 83, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "dir+": 242, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "dir-": 159, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "dn": 5826, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "dn+": 9069, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "dn-": 3243, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "cap": 0, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "cap+": 0, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "cap-": 0, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "rss": 64468, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "heap": 273166336 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.876 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_server": { 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_client_request": 0, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "dispatch_server_request": 0, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_request": 0, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_client_session": 0, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "handle_peer_request": 0, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "req_create_latency": { 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getattr_latency": { 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.877 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getfilelock_latency": { 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "req_link_latency": { 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookup_latency": { 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.878 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookuphash_latency": { 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupino_latency": { 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupname_latency": { 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.879 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupparent_latency": { 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lookupsnap_latency": { 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "req_lssnap_latency": { 2024-09-11T00:33:18.880 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mkdir_latency": { 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mknod_latency": { 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.881 INFO:teuthology.orchestra.run.smithi117.stdout: "req_mksnap_latency": { 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "req_open_latency": { 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "req_readdir_latency": { 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.882 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rename_latency": { 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "req_renamesnap_latency": { 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "req_snapdiff_latency": { 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.883 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmdir_latency": { 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmsnap_latency": { 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "req_rmxattr_latency": { 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.884 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setattr_latency": { 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setdirlayout_latency": { 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.885 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setfilelock_latency": { 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setlayout_latency": { 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "req_setxattr_latency": { 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "req_symlink_latency": { 2024-09-11T00:33:18.886 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "req_unlink_latency": { 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "req_getvxattr_latency": { 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.887 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_sessions": { 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "session_count": 1, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "session_add": 2, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "session_remove": 1, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_open": 1, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "sessions_stale": 0, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "total_load": 0, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "average_load": 0, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "avg_session_uptime": 2091, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "mdthresh_evicted": 0 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-11T00:33:18.888 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:18.889 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:18.890 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 206441, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 63, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:18.891 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 8856, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 166, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 4100680, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 22529, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.892 INFO:teuthology.orchestra.run.smithi117.stdout: "objecter": { 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_active": 0, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_laggy": 0, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send": 2139, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_send_bytes": 21, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_resend": 3, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_reply": 2139, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_latency": { 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2139, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 1.223974991, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000572218 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "op_inflight": 0, 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "oplen_avg": { 2024-09-11T00:33:18.893 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 2139, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 2140 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "op": 2139, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "op_r": 2139, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "op_w": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "op_rmw": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "op_pg": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_stat": 1354, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_create": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_read": 783, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_write": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writefull": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_writesame": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_append": 0, 2024-09-11T00:33:18.894 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_zero": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_truncate": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_delete": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_mapext": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_sparse_read": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_clonerange": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_getxattr": 1, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_setxattr": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_rmxattr": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_call": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_watch": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_notify": 0, 2024-09-11T00:33:18.895 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "osdop_other": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_active": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_send": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_resend": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "linger_ping": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_active": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_send": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "poolop_resend": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_active": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_send": 0, 2024-09-11T00:33:18.896 INFO:teuthology.orchestra.run.smithi117.stdout: "poolstat_resend": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_active": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_send": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "statfs_resend": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "command_active": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "command_send": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "command_resend": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "map_epoch": 92, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "map_full": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "map_inc": 7, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_sessions": 7, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_open": 7, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_session_close": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_laggy": 0, 2024-09-11T00:33:18.897 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_wr": 0, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_rd": 2, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_del": 0 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "oft": { 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_objs": 0, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_updates": 0, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "omap_total_removes": 0 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "purge_queue": { 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops": 0, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing": 0, 2024-09-11T00:33:18.898 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executing_high_water": 0, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed_ops": 0, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_executed": 0, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "pq_item_in_journal": 0 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 2391, 2024-09-11T00:33:18.899 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 34849903, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 2391, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2391, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 34849903, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.900 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_bytes": { 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2139, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 33838068, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2139, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 33838068, 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:18.901 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-objecter_ops": { 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 1024, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 2139, 2024-09-11T00:33:18.902 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 2139, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 2139, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 2139, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:18.903 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.904 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-write_buf_throttle-0x561762163040": { 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 3758096384, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 0, 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:18.905 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout: "trackedop": { 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout: "slow_ops_count": 0 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:18.906 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-11T00:33:19.817 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-11T00:33:19.817 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-11T00:33:19.818 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-11T00:33:19.818 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-09-11T00:33:19.818 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-11T00:33:19.818 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T00:33:19.819 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T00:33:20.064 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T00:33:20.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:20 smithi117 ceph-mon[35525]: pgmap v708: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 0, 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 2, 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 0, 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 0, 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 0, 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.000953136, 2024-09-11T00:33:20.597 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000161186, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.598 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 227, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 222, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 528528, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 136032, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 2, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 1, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.035975850, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.017946626, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.023270921, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 222, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.015786404, 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000071109 2024-09-11T00:33:20.599 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 528528, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 136032 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_messages": 1, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages": 168, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_bytes": 160, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_bytes": 109808, 2024-09-11T00:33:20.600 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_created_connections": 3, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_active_connections": 2, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_total_time": 0.011965604, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_send_time": 0.009669648, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_recv_time": 0.000728584, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 166, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.008124548, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000048943 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:20.601 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "msgr_send_encrypted_bytes": 109808 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "cct": { 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "total_workers": 0, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "unhealthy_workers": 0 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "mempool": { 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:20.602 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:20.603 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:20.604 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_items": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_bytes": 148945, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_anon_items": 29, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_bytes": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_items": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:20.605 INFO:teuthology.orchestra.run.smithi117.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_bytes": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_items": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "pgmap_items": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_bytes": 968, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "mds_co_items": 2, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_1_items": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: "unittest_2_items": 0 2024-09-11T00:33:20.606 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "val": 0, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "max": 104857600, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "get_started": 0, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "get": 224, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "get_sum": 494002, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "get_or_fail_success": 224, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "take": 0, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "take_sum": 0, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "put": 224, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "put_sum": 494002, 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "wait": { 2024-09-11T00:33:20.607 INFO:teuthology.orchestra.run.smithi117.stdout: "avgcount": 0, 2024-09-11T00:33:20.608 INFO:teuthology.orchestra.run.smithi117.stdout: "sum": 0.000000000, 2024-09-11T00:33:20.608 INFO:teuthology.orchestra.run.smithi117.stdout: "avgtime": 0.000000000 2024-09-11T00:33:20.608 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:20.608 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-11T00:33:20.608 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-11T00:33:20.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:20 smithi103 ceph-mon[30578]: pgmap v708: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-11T00:33:20.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:20 smithi183 ceph-mon[35934]: pgmap v708: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 7 op/s 2024-09-11T00:33:21.306 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-11T00:33:21.306 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-11T00:33:21.306 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-11T00:33:21.306 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-11T00:33:21.306 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-09-11T00:33:21.307 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T00:33:21.307 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T00:33:21.569 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:33:22.092 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-11T00:33:22.092 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:22.092 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 7427, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 12173, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2339743, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 10727492, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 13, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.585001524, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.341635309, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.225920464, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.043543826, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 12172, 2024-09-11T00:33:22.093 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.773031716, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000063509 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7757, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002155809, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000277 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1475, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1640, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 407605, 2024-09-11T00:33:22.094 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 4126772, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 11, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 9, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.146593182, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.069269296, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.027361545, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.039208365, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1636, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.163433459, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000099898 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-11T00:33:22.095 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000156, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000078 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 354416 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6812, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 6679, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4622126, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 2334778, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-11T00:33:22.096 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.507082385, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.251425215, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.273132631, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.028486640, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6680, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.370186664, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000055417 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11005, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001920371, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000174 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.097 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 649040, 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 140928 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 684, 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.612494264, 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002357447 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2904, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.355400618, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000122383 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 4145, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 4015, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4015, 2024-09-11T00:33:22.099 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 8.616045702, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002145964 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 129, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 6, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 27, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 5, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 6, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 16, 2024-09-11T00:33:22.100 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 19, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 286, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 1, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 1, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 4, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 6896, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 5660, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 129, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 44, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 1, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-11T00:33:22.101 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 52, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 181, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 52, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 1784, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:33:22.102 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 1402, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 1954, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 1708, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 294, 2024-09-11T00:33:22.103 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 43, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 3371 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 112, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 528, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 279, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 398, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 160, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 4, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 1624, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 1620, 2024-09-11T00:33:22.104 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 7, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 9225, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 52, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-11T00:33:22.105 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_started": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_succeeded": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_write_failed": 0 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 4602, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4602, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 6, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-11T00:33:22.106 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 6, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 11962702, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 291, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.750229484, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006014534 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-11T00:33:22.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 28, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 1701, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 1673, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 34, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 349, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 315, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 180, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 3856, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 3676, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 1, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 4271, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 4270, 2024-09-11T00:33:22.108 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 70588, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 282976256 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 4626, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 14, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 4145, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 8, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 184, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 121, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.273681524, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002261830 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-11T00:33:22.109 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-11T00:33:22.110 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2063, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.624386786, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000302659 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000220002, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000220002 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.111 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.112 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4, 2024-09-11T00:33:22.113 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004940826, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001235206 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.114 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 77, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.023171776, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000300932 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 123, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.126695039, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001030040 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5.421385710, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 2.710692855 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.115 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 26, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.456228890, 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.017547265 2024-09-11T00:33:22.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.117 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.118 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.119 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1598, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.685335149, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001054652 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-11T00:33:22.120 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 1, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 0, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 3026, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 3026, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 589, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:22.121 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:22.122 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:22.123 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 288901, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 112, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:22.124 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8856, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 166, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 373352, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 3631, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-11T00:33:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3711, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 7995285, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3711, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3711, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 32.849102845, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.008851819 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3711, 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3816 2024-09-11T00:33:22.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3711, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 39, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 3672, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 33, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 324, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 33, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-11T00:33:22.127 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 3234, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 5, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-11T00:33:22.128 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-11T00:33:22.129 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 92, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 7, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 119, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 12, 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 26 2024-09-11T00:33:22.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 19, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 11, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 3221, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 1620, 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-11T00:33:22.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 15710, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 6032775, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 15710, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 15710, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 6032775, 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:22.132 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:22.133 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3711, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 8171593, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3711, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 8171593, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:22.134 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3711, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3711, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3711, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3711, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:22.135 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 1620, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 176404, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 1620, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 33, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 176404, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x559f208dedc0": { 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4602, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 7768398, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4602, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 291, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 7768398, 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:22.137 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 9232 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:22.138 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-11T00:33:22.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:22 smithi103 ceph-mon[30578]: pgmap v709: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T00:33:22.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:22 smithi117 ceph-mon[35525]: pgmap v709: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T00:33:22.723 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:22 smithi183 ceph-mon[35934]: pgmap v709: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-11T00:33:22.932 INFO:tasks.check_counter:Daemon mds.c mds.exported=52 2024-09-11T00:33:22.932 INFO:tasks.check_counter:Daemon mds.c mds.imported=52 2024-09-11T00:33:22.932 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=112 2024-09-11T00:33:22.932 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=528 2024-09-11T00:33:22.932 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-11T00:33:22.932 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T00:33:22.932 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T00:33:23.182 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:33:23.690 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-11T00:33:23.690 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:23.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 94, 2024-09-11T00:33:23.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 96, 2024-09-11T00:33:23.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 31935, 2024-09-11T00:33:23.690 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 29902, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.013633920, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.005187228, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.003603586, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.003094497, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 94, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.008361329, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000088950 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.691 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 229, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 248, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2902633, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 196809, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 5, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.023885362, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.008785201, 2024-09-11T00:33:23.692 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.009012969, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.004801440, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 247, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.016919257, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000068499 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 125520 2024-09-11T00:33:23.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 41, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 38, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 191763, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 18292, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.005691929, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.002205772, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.009821418, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000054864, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 37, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003805531, 2024-09-11T00:33:23.694 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000102852 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 190368, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 17328 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-11T00:33:23.695 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 540, 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.013138465, 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000024330 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.696 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-11T00:33:23.697 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 20, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 9, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 11, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 7, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-11T00:33:23.698 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 2, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-11T00:33:23.699 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-11T00:33:23.700 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-11T00:33:23.701 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:33:23.702 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_started": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_succeeded": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_write_failed": 0 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-11T00:33:23.703 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 1300, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 6, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 82594641, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 82616259, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 82612837, 2024-09-11T00:33:23.704 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 1300 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 9, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 22, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 13, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 8, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 11, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 3, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 20, 2024-09-11T00:33:23.705 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 39, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 19, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 46964, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 248238080 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.706 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.707 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.708 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.709 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.710 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-11T00:33:23.711 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.714 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.715 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.716 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-11T00:33:23.717 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 1, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 0, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 0, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2102, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:23.718 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:23.719 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:23.720 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:23.721 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-11T00:33:23.721 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:23.721 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:23.721 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:23.721 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:23.721 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 196519, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 49, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8856, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 166, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:23.722 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 45013, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 204, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 321, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 26, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 4, 2024-09-11T00:33:23.723 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 321, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 321, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.235993541, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000735182 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 321, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 322 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 321, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 321, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-11T00:33:23.724 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 182, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 136, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-11T00:33:23.725 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 2, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-11T00:33:23.726 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 0, 2024-09-11T00:33:23.727 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 6, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-11T00:33:23.728 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 360, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 3089017, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:23.729 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 360, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 360, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3089017, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:23.730 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 321, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2842484, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 321, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2842484, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-11T00:33:23.731 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 321, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 321, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 321, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 321, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.732 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:23.733 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:23.734 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:23.734 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-11T00:33:23.734 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-11T00:33:23.734 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:23.734 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.734 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.735 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.735 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x559be518ac80": { 2024-09-11T00:33:23.735 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:23.735 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:23.736 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:23.737 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-09-11T00:33:23.737 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-11T00:33:23.737 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:23.737 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-11T00:33:24.521 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:24 smithi183 ceph-mon[35934]: pgmap v710: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:24.547 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T00:33:24.548 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T00:33:24.548 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-11T00:33:24.548 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-11T00:33:24.548 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-11T00:33:24.548 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T00:33:24.548 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T00:33:24.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:24 smithi103 ceph-mon[30578]: pgmap v710: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:24.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:24 smithi117 ceph-mon[35525]: pgmap v710: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:24.792 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 7206, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 6808, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 6124732, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 22798489, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 9, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.608386246, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.307094873, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.236168259, 2024-09-11T00:33:25.261 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.088757476, 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6806, 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.633901114, 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000093138 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7661, 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001415679, 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000184 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.262 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5678, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 5182, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1600625, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 4366636, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 11, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.438343495, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.229307386, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.153715835, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.065359264, 2024-09-11T00:33:25.263 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5181, 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.602242977, 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000116240 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5049, 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000728022, 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000144 2024-09-11T00:33:25.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 4736, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 359584 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 20766, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 34695, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 7673935, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 24308782, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 12, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 1.541616249, 2024-09-11T00:33:25.265 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.943306764, 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.749463837, 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.101843298, 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 34694, 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 7.216850770, 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000208014 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.266 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 21552, 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.006480249, 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000300 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 652272, 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 139632 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-11T00:33:25.267 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 628, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 7.802656580, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.012424612 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7683, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.224271756, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000159348 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 9775, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 8906, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8906, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 11.303884146, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001269243 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 869, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 9, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-11T00:33:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 20, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 1, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 1, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 387, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 262, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 113, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 12, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 12, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 1517, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 3, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 13017, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 11985, 2024-09-11T00:33:25.270 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 836, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 44, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 2, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 26, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 1441, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 107, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 4602, 2024-09-11T00:33:25.271 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-11T00:33:25.272 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 56, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 1851, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 5866, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 6986, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 3768, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 162, 2024-09-11T00:33:25.273 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 7315 2024-09-11T00:33:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-11T00:33:25.274 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 282, 2024-09-11T00:33:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 156, 2024-09-11T00:33:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 99, 2024-09-11T00:33:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 140, 2024-09-11T00:33:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 491, 2024-09-11T00:33:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 2, 2024-09-11T00:33:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-11T00:33:25.277 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 4304, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 4302, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 7, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 16044, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 26, 2024-09-11T00:33:25.278 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 2, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_started": 0, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_succeeded": 0, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "uninline_write_failed": 0 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 1, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 15464, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 15464, 2024-09-11T00:33:25.279 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 21, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 21, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 3, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 38024679, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-11T00:33:25.280 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 331, 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 10.417832205, 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.031473813 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 19, 2024-09-11T00:33:25.281 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 5492, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 5473, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 87, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 291, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 204, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 387, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 9138, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 8751, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-11T00:33:25.282 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 9796, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 9796, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 96720, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 327389184 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 10162, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 79, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 9775, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 37, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 211, 2024-09-11T00:33:25.283 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1309, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.661007197, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001268913 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.035234052, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003914894 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.284 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3006, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.214751624, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000404108 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.285 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.286 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-11T00:33:25.287 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-11T00:33:25.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.261095472, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.261095472 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001241306, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000206884 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 266, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.258876820, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000973221 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003728799, 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001242933 2024-09-11T00:33:25.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 36, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.143062817, 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.059529522 2024-09-11T00:33:25.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.293 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4270, 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5.724886059, 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001340722 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-11T00:33:25.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 5582, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 5582, 2024-09-11T00:33:25.296 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 519, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:25.297 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:25.298 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 287279, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 99, 2024-09-11T00:33:25.299 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 8856, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 166, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 446232, 2024-09-11T00:33:25.300 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 4418, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 9291, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 34897264, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 30, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 9291, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9291, 2024-09-11T00:33:25.301 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 78.169858705, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.008413503 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9291, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 9395 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 9291, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 74, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 9217, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-11T00:33:25.302 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 65, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 396, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 46, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 1, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 8689, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 3, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-09-11T00:33:25.303 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-11T00:33:25.304 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-11T00:33:25.305 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 92, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 7, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 118, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 18, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 29 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-11T00:33:25.306 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 65, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 9, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 8661, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 4302, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:25.307 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 33646, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 12624880, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 33646, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 33646, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 12624880, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 9291, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 35365822, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 9291, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 35365822, 2024-09-11T00:33:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 9291, 2024-09-11T00:33:25.310 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 9291, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 9291, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 9291, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4302, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 468702, 2024-09-11T00:33:25.311 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4302, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 65, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 468702, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.312 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x5593747ccdc0": { 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 15464, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 33830375, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 15464, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 331, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 33830375, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:25.313 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-09-11T00:33:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 16051 2024-09-11T00:33:25.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:25.314 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-11T00:33:26.143 INFO:tasks.check_counter:Daemon mds.i mds.exported=26 2024-09-11T00:33:26.143 INFO:tasks.check_counter:Daemon mds.i mds.imported=107 2024-09-11T00:33:26.143 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=282 2024-09-11T00:33:26.143 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=156 2024-09-11T00:33:26.143 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-09-11T00:33:26.143 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T00:33:26.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T00:33:26.389 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T00:33:26.509 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:26 smithi183 ceph-mon[35934]: pgmap v711: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:26.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:26 smithi103 ceph-mon[30578]: pgmap v711: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:26.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:26 smithi117 ceph-mon[35525]: pgmap v711: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 0, 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2, 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 0, 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 0, 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.001058706, 2024-09-11T00:33:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000242431, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 171, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 3152, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 110432, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.014387438, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.010053899, 2024-09-11T00:33:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002667174, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 170, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.013579183, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000079877 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 110432 2024-09-11T00:33:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 225, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 222, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 526080, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 136864, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.033933248, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.017422878, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.020534709, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 221, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.012832564, 2024-09-11T00:33:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000058065 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 526080, 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 136864 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-11T00:33:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T00:33:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T00:33:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 149007, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 30, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-11T00:33:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 968, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-11T00:33:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 226, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 494262, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 226, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 226, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 494262, 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-11T00:33:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-11T00:33:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-11T00:33:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-11T00:33:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-11T00:33:27.022 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-11T00:33:27.759 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-11T00:33:27.759 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-11T00:33:27.759 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-11T00:33:27.759 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-11T00:33:27.759 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-09-11T00:33:27.760 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T00:33:27.770 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T00:33:27.771 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:33:27.771 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:33:27.808 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-09-11T00:33:27.809 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T00:33:27.886 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T00:33:28.305 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:28.628 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:28 smithi103 ceph-mon[30578]: pgmap v712: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:28.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:28 smithi117 ceph-mon[35525]: pgmap v712: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:28.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:28 smithi183 ceph-mon[35934]: pgmap v712: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-11T00:33:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 -- 172.21.15.103:0/2960675582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c10a9c0 msgr2=0x7fd99c10ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/2960675582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c10a9c0 0x7fd99c10ae00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd988009860 tx=0x7fd98802a340 comp rx=0 tx=0).stop 2024-09-11T00:33:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 -- 172.21.15.103:0/2960675582 shutdown_connections 2024-09-11T00:33:28.918 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/2960675582 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c10b340 0x7fd99c111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/2960675582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c10a9c0 0x7fd99c10ae00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/2960675582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99c104fb0 0x7fd99c105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 -- 172.21.15.103:0/2960675582 >> 172.21.15.103:0/2960675582 conn(0x7fd99c100a20 msgr2=0x7fd99c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 -- 172.21.15.103:0/2960675582 shutdown_connections 2024-09-11T00:33:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.915+0000 7fd9a3e00640 1 -- 172.21.15.103:0/2960675582 wait complete. 2024-09-11T00:33:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.916+0000 7fd9a3e00640 1 Processor -- start 2024-09-11T00:33:28.919 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.916+0000 7fd9a3e00640 1 -- start start 2024-09-11T00:33:28.920 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c104fb0 0x7fd99c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a3e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c10a9c0 0x7fd99c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a3e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99c10b340 0x7fd99c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a3e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c114f60 con 0x7fd99c104fb0 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a3e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c114de0 con 0x7fd99c10b340 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a3e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c1150e0 con 0x7fd99c10a9c0 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c104fb0 0x7fd99c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c104fb0 0x7fd99c19b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36340/0 (socket says 172.21.15.103:36340) 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 -- 172.21.15.103:0/1113353419 learned_addr learned my addr 172.21.15.103:0/1113353419 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99c10b340 0x7fd99c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 -- 172.21.15.103:0/1113353419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c10a9c0 msgr2=0x7fd99c19ba50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c10a9c0 0x7fd99c19ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 -- 172.21.15.103:0/1113353419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99c10b340 msgr2=0x7fd99c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:28.921 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99c10b340 0x7fd99c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1400640 1 -- 172.21.15.103:0/1113353419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c1a15c0 con 0x7fd99c104fb0 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.917+0000 7fd9a1e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99c10b340 0x7fd99c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.918+0000 7fd9a1400640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c104fb0 0x7fd99c19b510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd99000eb90 tx=0x7fd99000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.918+0000 7fd99a000640 1 -- 172.21.15.103:0/1113353419 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990004d20 con 0x7fd99c104fb0 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.918+0000 7fd99a000640 1 -- 172.21.15.103:0/1113353419 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99000ee90 con 0x7fd99c104fb0 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.918+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c1a18b0 con 0x7fd99c104fb0 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.918+0000 7fd99a000640 1 -- 172.21.15.103:0/1113353419 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990010670 con 0x7fd99c104fb0 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.918+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1a7f70 con 0x7fd99c104fb0 2024-09-11T00:33:28.922 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.919+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c105390 con 0x7fd99c104fb0 2024-09-11T00:33:28.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.921+0000 7fd99a000640 1 -- 172.21.15.103:0/1113353419 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fd990005020 con 0x7fd99c104fb0 2024-09-11T00:33:28.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.923+0000 7fd99a000640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9840b5360 0x7fd9840b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:28.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.923+0000 7fd99a000640 1 -- 172.21.15.103:0/1113353419 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fd9900e66e0 con 0x7fd99c104fb0 2024-09-11T00:33:28.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.923+0000 7fd9a0a00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9840b5360 0x7fd9840b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:28.926 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.924+0000 7fd9a0a00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9840b5360 0x7fd9840b7820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd98802a860 tx=0x7fd988000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:28.929 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:28.927+0000 7fd99a000640 1 -- 172.21.15.103:0/1113353419 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd9900ad1b0 con 0x7fd99c104fb0 2024-09-11T00:33:29.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.124+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd99c02e5c0 con 0x7fd99c104fb0 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.124+0000 7fd99a000640 1 -- 172.21.15.103:0/1113353419 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+598 (secure 0 0 0) 0x7fd9900a8020 con 0x7fd99c104fb0 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/489365644 2024-09-12T00:12:26.257361+0000 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1436162684 2024-09-12T00:12:26.257361+0000 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/2651148903 2024-09-12T00:12:26.257361+0000 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/817530946 2024-09-12T00:12:26.257361+0000 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/2651148903 2024-09-12T00:12:26.257361+0000 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/1510088401 2024-09-12T00:11:57.620215+0000 2024-09-11T00:33:29.127 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6801/477475422 2024-09-12T00:11:57.620215+0000 2024-09-11T00:33:29.128 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:6800/477475422 2024-09-12T00:11:57.620215+0000 2024-09-11T00:33:29.128 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/3506454988 2024-09-12T00:11:57.620215+0000 2024-09-11T00:33:29.128 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.15.103:0/2690135380 2024-09-12T00:11:57.620215+0000 2024-09-11T00:33:29.128 INFO:teuthology.orchestra.run.smithi103.stderr:listed 10 entries 2024-09-11T00:33:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9840b5360 msgr2=0x7fd9840b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9840b5360 0x7fd9840b7820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd98802a860 tx=0x7fd988000bc0 comp rx=0 tx=0).stop 2024-09-11T00:33:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c104fb0 msgr2=0x7fd99c19b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c104fb0 0x7fd99c19b510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd99000eb90 tx=0x7fd99000c6a0 comp rx=0 tx=0).stop 2024-09-11T00:33:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 shutdown_connections 2024-09-11T00:33:29.129 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fd9840b5360 0x7fd9840b7820 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd99c10b340 0x7fd99c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd99c10a9c0 0x7fd99c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 --2- 172.21.15.103:0/1113353419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd99c104fb0 0x7fd99c19b510 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 >> 172.21.15.103:0/1113353419 conn(0x7fd99c100a20 msgr2=0x7fd99c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 shutdown_connections 2024-09-11T00:33:29.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:29.127+0000 7fd9a3e00640 1 -- 172.21.15.103:0/1113353419 wait complete. 2024-09-11T00:33:29.560 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:29 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1113353419' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:33:29.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:29 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1113353419' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:33:29.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:29 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1113353419' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-11T00:33:29.956 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T00:33:29.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:33:29.957 DEBUG:teuthology.orchestra.run.smithi103:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T00:33:30.062 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:33:30.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:33:30.062 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T00:33:30.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:33:30.138 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-09-11T00:33:30.189 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T00:33:30.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:33:30.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T00:33:30.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T00:33:30.247 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-09-11T00:33:30.522 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:30 smithi103 ceph-mon[30578]: pgmap v713: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-11T00:33:30.579 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T00:33:30.588 INFO:tasks.cephadm:Teardown begin 2024-09-11T00:33:30.589 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T00:33:30.589 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:33:30.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:30 smithi117 ceph-mon[35525]: pgmap v713: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-11T00:33:30.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:30 smithi183 ceph-mon[35934]: pgmap v713: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-11T00:33:30.963 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:31.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 -- 172.21.15.103:0/2460407863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054102c10 msgr2=0x7f1054102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:31.604 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 --2- 172.21.15.103:0/2460407863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054102c10 0x7f1054102ff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1044009900 tx=0x7f104402a410 comp rx=0 tx=0).stop 2024-09-11T00:33:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 -- 172.21.15.103:0/2460407863 shutdown_connections 2024-09-11T00:33:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 --2- 172.21.15.103:0/2460407863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1054108fa0 0x7f105410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 --2- 172.21.15.103:0/2460407863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 0x7f1054108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 --2- 172.21.15.103:0/2460407863 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054102c10 0x7f1054102ff0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 -- 172.21.15.103:0/2460407863 >> 172.21.15.103:0/2460407863 conn(0x7f10540fe660 msgr2=0x7f1054100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 -- 172.21.15.103:0/2460407863 shutdown_connections 2024-09-11T00:33:31.605 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.601+0000 7f1059000640 1 -- 172.21.15.103:0/2460407863 wait complete. 2024-09-11T00:33:31.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.602+0000 7f1059000640 1 Processor -- start 2024-09-11T00:33:31.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1059000640 1 -- start start 2024-09-11T00:33:31.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1059000640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1054102c10 0x7f105419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:31.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1059000640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 0x7f105419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1059000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054108fa0 0x7f10541a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1059000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1054112c70 con 0x7f1054108fa0 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1059000640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f1054112af0 con 0x7f1054108620 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1059000640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1054112df0 con 0x7f1054102c10 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1052a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 0x7f105419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1052a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 0x7f105419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:36686/0 (socket says 172.21.15.103:36686) 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1053e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054108fa0 0x7f10541a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.603+0000 7f1052a00640 1 -- 172.21.15.103:0/26101487 learned_addr learned my addr 172.21.15.103:0/26101487 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.604+0000 7f1053400640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1054102c10 0x7f105419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.604+0000 7f1052a00640 1 -- 172.21.15.103:0/26101487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1054102c10 msgr2=0x7f105419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.604+0000 7f1052a00640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1054102c10 0x7f105419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.604+0000 7f1052a00640 1 -- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054108fa0 msgr2=0x7f10541a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.604+0000 7f1052a00640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054108fa0 0x7f10541a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.604+0000 7f1052a00640 1 -- 172.21.15.103:0/26101487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10541a1300 con 0x7f1054108620 2024-09-11T00:33:31.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.604+0000 7f1052a00640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 0x7f105419b790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f104000e9f0 tx=0x7f104000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:31.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.605+0000 7f104be00640 1 -- 172.21.15.103:0/26101487 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000cd80 con 0x7f1054108620 2024-09-11T00:33:31.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.605+0000 7f104be00640 1 -- 172.21.15.103:0/26101487 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10400108e0 con 0x7f1054108620 2024-09-11T00:33:31.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.605+0000 7f104be00640 1 -- 172.21.15.103:0/26101487 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040018980 con 0x7f1054108620 2024-09-11T00:33:31.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.605+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10541a15f0 con 0x7f1054108620 2024-09-11T00:33:31.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.605+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10541a7cb0 con 0x7f1054108620 2024-09-11T00:33:31.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.606+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1054102ff0 con 0x7f1054108620 2024-09-11T00:33:31.611 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.609+0000 7f104be00640 1 -- 172.21.15.103:0/26101487 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f1040010430 con 0x7f1054108620 2024-09-11T00:33:31.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.610+0000 7f104be00640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10380b5410 0x7f10380b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:31.612 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.610+0000 7f104be00640 1 -- 172.21.15.103:0/26101487 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f10400e6710 con 0x7f1054108620 2024-09-11T00:33:31.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.611+0000 7f1053400640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10380b5410 0x7f10380b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:31.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.611+0000 7f104be00640 1 -- 172.21.15.103:0/26101487 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f10400acb90 con 0x7f1054108620 2024-09-11T00:33:31.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.611+0000 7f1053400640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10380b5410 0x7f10380b78d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1044005ab0 tx=0x7f1044005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:31.843 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.841+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f105402e5c0 con 0x7f1054108620 2024-09-11T00:33:31.844 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.842+0000 7f104be00640 1 -- 172.21.15.103:0/26101487 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11937 (secure 0 0 0) 0x7f10400b15c0 con 0x7f1054108620 2024-09-11T00:33:31.845 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:33:31.845 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":47,"btime":"2024-09-11T00:32:58:220802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:32:57.689346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26323":{"gid":26323,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4278110475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4278110475},{"type":"v1","addr":"172.21.15.183:6835","nonce":4278110475}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:33:31.845 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 47 2024-09-11T00:33:31.847 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10380b5410 msgr2=0x7f10380b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10380b5410 0x7f10380b78d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1044005ab0 tx=0x7f1044005a20 comp rx=0 tx=0).stop 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 msgr2=0x7f105419b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 0x7f105419b790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f104000e9f0 tx=0x7f104000eeb0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 shutdown_connections 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f10380b5410 0x7f10380b78d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1054108fa0 0x7f10541a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1054108620 0x7f105419b790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 --2- 172.21.15.103:0/26101487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1054102c10 0x7f105419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.845+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 >> 172.21.15.103:0/26101487 conn(0x7f10540fe660 msgr2=0x7f1054104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.846+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 shutdown_connections 2024-09-11T00:33:31.848 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:31.846+0000 7f1059000640 1 -- 172.21.15.103:0/26101487 wait complete. 2024-09-11T00:33:32.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:32 smithi103 ceph-mon[30578]: pgmap v714: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:33:32.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:32 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/26101487' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:33:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:32 smithi117 ceph-mon[35525]: pgmap v714: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:33:32.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:32 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/26101487' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:33:32.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T00:33:32.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:32 smithi183 ceph-mon[35934]: pgmap v714: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-11T00:33:32.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:32 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/26101487' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:33:33.099 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 -- 172.21.15.103:0/770727104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304077ff0 msgr2=0x7f0304113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 --2- 172.21.15.103:0/770727104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304077ff0 0x7f0304113450 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009900 tx=0x7f02f002a410 comp rx=0 tx=0).stop 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 -- 172.21.15.103:0/770727104 shutdown_connections 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 --2- 172.21.15.103:0/770727104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0304113990 0x7f0304115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 --2- 172.21.15.103:0/770727104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304077ff0 0x7f0304113450 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 --2- 172.21.15.103:0/770727104 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f03040776d0 0x7f0304077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 -- 172.21.15.103:0/770727104 >> 172.21.15.103:0/770727104 conn(0x7f0304100a50 msgr2=0x7f0304102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:33.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 -- 172.21.15.103:0/770727104 shutdown_connections 2024-09-11T00:33:33.871 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.867+0000 7f030c200640 1 -- 172.21.15.103:0/770727104 wait complete. 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.868+0000 7f030c200640 1 Processor -- start 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.868+0000 7f030c200640 1 -- start start 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f030c200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03040776d0 0x7f030419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f030c200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0304077ff0 0x7f030419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f030c200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304113990 0x7f03041a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f030c200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03041193c0 con 0x7f0304113990 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f030c200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f0304119240 con 0x7f03040776d0 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f030c200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0304119540 con 0x7f0304077ff0 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f0309800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03040776d0 0x7f030419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f0309800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03040776d0 0x7f030419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:38142/0 (socket says 172.21.15.103:38142) 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f0309800640 1 -- 172.21.15.103:0/4170234975 learned_addr learned my addr 172.21.15.103:0/4170234975 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:33.872 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f030a200640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304113990 0x7f03041a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.869+0000 7f0308e00640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0304077ff0 0x7f030419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0309800640 1 -- 172.21.15.103:0/4170234975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0304077ff0 msgr2=0x7f030419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0309800640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0304077ff0 0x7f030419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0309800640 1 -- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304113990 msgr2=0x7f03041a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0309800640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304113990 0x7f03041a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0309800640 1 -- 172.21.15.103:0/4170234975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03041a5960 con 0x7f03040776d0 2024-09-11T00:33:33.873 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0309800640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03040776d0 0x7f030419f8b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02f800ebc0 tx=0x7f02f800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:33.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0302000640 1 -- 172.21.15.103:0/4170234975 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8010630 con 0x7f03040776d0 2024-09-11T00:33:33.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03041a5c50 con 0x7f03040776d0 2024-09-11T00:33:33.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0302000640 1 -- 172.21.15.103:0/4170234975 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f8010c80 con 0x7f03040776d0 2024-09-11T00:33:33.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f0302000640 1 -- 172.21.15.103:0/4170234975 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8018b60 con 0x7f03040776d0 2024-09-11T00:33:33.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.870+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03041ac4e0 con 0x7f03040776d0 2024-09-11T00:33:33.874 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.871+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0304078be0 con 0x7f03040776d0 2024-09-11T00:33:33.876 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.875+0000 7f0302000640 1 -- 172.21.15.103:0/4170234975 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f02f80107d0 con 0x7f03040776d0 2024-09-11T00:33:33.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.876+0000 7f0302000640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f02ec0b52e0 0x7f02ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:33.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.876+0000 7f0302000640 1 -- 172.21.15.103:0/4170234975 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f02f80e6b20 con 0x7f03040776d0 2024-09-11T00:33:33.878 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.876+0000 7f0308e00640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f02ec0b52e0 0x7f02ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:33.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.877+0000 7f0308e00640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f02ec0b52e0 0x7f02ec0b77a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000c00 tx=0x7f02f0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:33.879 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:33.877+0000 7f0302000640 1 -- 172.21.15.103:0/4170234975 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f02f80ad5f0 con 0x7f03040776d0 2024-09-11T00:33:34.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.098+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03041ac6b0 con 0x7f03040776d0 2024-09-11T00:33:34.101 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.099+0000 7f0302000640 1 -- 172.21.15.103:0/4170234975 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11937 (secure 0 0 0) 0x7f02f80acef0 con 0x7f03040776d0 2024-09-11T00:33:34.101 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:33:34.101 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":47,"btime":"2024-09-11T00:32:58:220802+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14580,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/1632166659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":1632166659},{"type":"v1","addr":"172.21.15.103:6835","nonce":1632166659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14592,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/3157272512","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":3157272512},{"type":"v1","addr":"172.21.15.103:6837","nonce":3157272512}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/3371048612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":3371048612},{"type":"v1","addr":"172.21.15.103:6839","nonce":3371048612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6841/4229734989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6840","nonce":4229734989},{"type":"v1","addr":"172.21.15.103:6841","nonce":4229734989}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24566,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/928029640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":928029640},{"type":"v1","addr":"172.21.15.117:6833","nonce":928029640}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24571,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/1084194007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":1084194007},{"type":"v1","addr":"172.21.15.183:6833","nonce":1084194007}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T00:20:28.229329+0000","modified":"2024-09-11T00:32:57.689346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24656,"mds_1":24643,"mds_2":24628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24599":{"gid":24599,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6835/2973217634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2973217634},{"type":"v1","addr":"172.21.15.117:6835","nonce":2973217634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24620":{"gid":24620,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.117:6837/2700270511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2700270511},{"type":"v1","addr":"172.21.15.117:6837","nonce":2700270511}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24628":{"gid":24628,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":23,"addr":"172.21.15.183:6837/335217865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":335217865},{"type":"v1","addr":"172.21.15.183:6837","nonce":335217865}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24643":{"gid":24643,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":21,"addr":"172.21.15.183:6839/424027451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":424027451},{"type":"v1","addr":"172.21.15.183:6839","nonce":424027451}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24656":{"gid":24656,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.117:6839/3769588373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6838","nonce":3769588373},{"type":"v1","addr":"172.21.15.117:6839","nonce":3769588373}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26323":{"gid":26323,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/4278110475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":4278110475},{"type":"v1","addr":"172.21.15.183:6835","nonce":4278110475}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24656,"qdb_cluster":[24628,24643,24656]},"id":1}]} 2024-09-11T00:33:34.102 INFO:teuthology.orchestra.run.smithi103.stderr:dumped fsmap epoch 47 2024-09-11T00:33:34.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f02ec0b52e0 msgr2=0x7f02ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f02ec0b52e0 0x7f02ec0b77a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f02f0000c00 tx=0x7f02f0031400 comp rx=0 tx=0).stop 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03040776d0 msgr2=0x7f030419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03040776d0 0x7f030419f8b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f02f800ebc0 tx=0x7f02f800c6a0 comp rx=0 tx=0).stop 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 shutdown_connections 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f02ec0b52e0 0x7f02ec0b77a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0304113990 0x7f03041a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0304077ff0 0x7f030419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 --2- 172.21.15.103:0/4170234975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03040776d0 0x7f030419f8b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 >> 172.21.15.103:0/4170234975 conn(0x7f0304100a50 msgr2=0x7f0304102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.101+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 shutdown_connections 2024-09-11T00:33:34.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:34.102+0000 7f030c200640 1 -- 172.21.15.103:0/4170234975 wait complete. 2024-09-11T00:33:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:34 smithi103 ceph-mon[30578]: pgmap v715: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T00:33:34.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:34 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/4170234975' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:33:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:34 smithi117 ceph-mon[35525]: pgmap v715: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T00:33:34.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:34 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/4170234975' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:33:34.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:34 smithi183 ceph-mon[35934]: pgmap v715: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-11T00:33:34.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:34 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/4170234975' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-11T00:33:34.994 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T00:33:35.370 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 -- 172.21.15.103:0/652784170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe234103c90 msgr2=0x7fe234104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 --2- 172.21.15.103:0/652784170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe234103c90 0x7fe234104070 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe224009900 tx=0x7fe22402a410 comp rx=0 tx=0).stop 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 -- 172.21.15.103:0/652784170 shutdown_connections 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 --2- 172.21.15.103:0/652784170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe23410d0f0 0x7fe23410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 --2- 172.21.15.103:0/652784170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2341045b0 0x7fe23410caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 --2- 172.21.15.103:0/652784170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe234103c90 0x7fe234104070 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 -- 172.21.15.103:0/652784170 >> 172.21.15.103:0/652784170 conn(0x7fe2340fd710 msgr2=0x7fe2340ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:36.043 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 -- 172.21.15.103:0/652784170 shutdown_connections 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.032+0000 7fe239e00640 1 -- 172.21.15.103:0/652784170 wait complete. 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.033+0000 7fe239e00640 1 Processor -- start 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.033+0000 7fe239e00640 1 -- start start 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe239e00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe234103c90 0x7fe23419b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe239e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2341045b0 0x7fe23419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe239e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe23410d0f0 0x7fe2341a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe239e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe234112d60 con 0x7fe23410d0f0 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe239e00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fe234112be0 con 0x7fe234103c90 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe239e00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe234112ee0 con 0x7fe2341045b0 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe234103c90 0x7fe23419b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe233400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe23410d0f0 0x7fe2341a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:36.044 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe234103c90 0x7fe23419b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:38158/0 (socket says 172.21.15.103:38158) 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 -- 172.21.15.103:0/636126424 learned_addr learned my addr 172.21.15.103:0/636126424 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe22be00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2341045b0 0x7fe23419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 -- 172.21.15.103:0/636126424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2341045b0 msgr2=0x7fe23419b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2341045b0 0x7fe23419b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 -- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe23410d0f0 msgr2=0x7fe2341a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe23410d0f0 0x7fe2341a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.034+0000 7fe232a00640 1 -- 172.21.15.103:0/636126424 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2341a1560 con 0x7fe234103c90 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.035+0000 7fe232a00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe234103c90 0x7fe23419b4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe224005940 tx=0x7fe224004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.035+0000 7fe238a00640 1 -- 172.21.15.103:0/636126424 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22403f030 con 0x7fe234103c90 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.035+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2341a17f0 con 0x7fe234103c90 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.035+0000 7fe238a00640 1 -- 172.21.15.103:0/636126424 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe224003aa0 con 0x7fe234103c90 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.035+0000 7fe238a00640 1 -- 172.21.15.103:0/636126424 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe224035070 con 0x7fe234103c90 2024-09-11T00:33:36.045 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.035+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2341a7f10 con 0x7fe234103c90 2024-09-11T00:33:36.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.036+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe234104070 con 0x7fe234103c90 2024-09-11T00:33:36.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.039+0000 7fe238a00640 1 -- 172.21.15.103:0/636126424 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fe22403ba90 con 0x7fe234103c90 2024-09-11T00:33:36.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.040+0000 7fe238a00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2180b5360 0x7fe2180b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:36.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.040+0000 7fe238a00640 1 -- 172.21.15.103:0/636126424 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7fe224106b00 con 0x7fe234103c90 2024-09-11T00:33:36.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.041+0000 7fe22be00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2180b5360 0x7fe2180b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:36.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.041+0000 7fe22be00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2180b5360 0x7fe2180b7820 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe23410f7c0 tx=0x7fe21c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:36.046 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.042+0000 7fe238a00640 1 -- 172.21.15.103:0/636126424 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe2240ccf80 con 0x7fe234103c90 2024-09-11T00:33:36.233 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.230+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2341a8200 con 0x7fe234103c90 2024-09-11T00:33:36.234 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.232+0000 7fe238a00640 1 -- 172.21.15.103:0/636126424 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24865 (secure 0 0 0) 0x7fe2240d19b0 con 0x7fe234103c90 2024-09-11T00:33:36.235 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-09-11T00:33:36.235 INFO:teuthology.orchestra.run.smithi103.stdout:{"epoch":92,"fsid":"40f10072-6fd2-11ef-bcea-c7b262605968","created":"2024-09-11T00:11:30.997477+0000","modified":"2024-09-11T00:31:53.221970+0000","last_up_change":"2024-09-11T00:19:28.528034+0000","last_in_change":"2024-09-11T00:19:05.721852+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T00:15:22.579961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T00:20:22.113281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T00:20:25.223289+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-11T00:20:35.424587+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b2d81a93-417a-401b-88db-d1512a1bb426","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6803","nonce":2122983422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6805","nonce":2122983422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6809","nonce":2122983422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2122983422},{"type":"v1","addr":"172.21.15.103:6807","nonce":2122983422}]},"public_addr":"172.21.15.103:6803/2122983422","cluster_addr":"172.21.15.103:6805/2122983422","heartbeat_back_addr":"172.21.15.103:6809/2122983422","heartbeat_front_addr":"172.21.15.103:6807/2122983422","state":["exists","up"]},{"osd":1,"uuid":"0a35cffa-4c5a-4c39-851e-563d54f29b49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6801","nonce":2016738362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6803","nonce":2016738362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6807","nonce":2016738362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":2016738362},{"type":"v1","addr":"172.21.15.117:6805","nonce":2016738362}]},"public_addr":"172.21.15.117:6801/2016738362","cluster_addr":"172.21.15.117:6803/2016738362","heartbeat_back_addr":"172.21.15.117:6807/2016738362","heartbeat_front_addr":"172.21.15.117:6805/2016738362","state":["exists","up"]},{"osd":2,"uuid":"fca0072a-688a-4c07-90b9-bd4bb84aa15f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6801","nonce":4126324003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6803","nonce":4126324003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6807","nonce":4126324003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":4126324003},{"type":"v1","addr":"172.21.15.183:6805","nonce":4126324003}]},"public_addr":"172.21.15.183:6801/4126324003","cluster_addr":"172.21.15.183:6803/4126324003","heartbeat_back_addr":"172.21.15.183:6807/4126324003","heartbeat_front_addr":"172.21.15.183:6805/4126324003","state":["exists","up"]},{"osd":3,"uuid":"39a7f76f-3edd-4144-b4c4-a3e2ca84001d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6811","nonce":2768325174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6813","nonce":2768325174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6817","nonce":2768325174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":2768325174},{"type":"v1","addr":"172.21.15.103:6815","nonce":2768325174}]},"public_addr":"172.21.15.103:6811/2768325174","cluster_addr":"172.21.15.103:6813/2768325174","heartbeat_back_addr":"172.21.15.103:6817/2768325174","heartbeat_front_addr":"172.21.15.103:6815/2768325174","state":["exists","up"]},{"osd":4,"uuid":"5f479678-1b4e-4d7b-9c3e-d2033337c993","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6809","nonce":392780746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6811","nonce":392780746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6815","nonce":392780746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":392780746},{"type":"v1","addr":"172.21.15.117:6813","nonce":392780746}]},"public_addr":"172.21.15.117:6809/392780746","cluster_addr":"172.21.15.117:6811/392780746","heartbeat_back_addr":"172.21.15.117:6815/392780746","heartbeat_front_addr":"172.21.15.117:6813/392780746","state":["exists","up"]},{"osd":5,"uuid":"3455e30e-a2f0-4bc5-9b98-ef3b8e2e939d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6809","nonce":660429502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6811","nonce":660429502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6815","nonce":660429502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":660429502},{"type":"v1","addr":"172.21.15.183:6813","nonce":660429502}]},"public_addr":"172.21.15.183:6809/660429502","cluster_addr":"172.21.15.183:6811/660429502","heartbeat_back_addr":"172.21.15.183:6815/660429502","heartbeat_front_addr":"172.21.15.183:6813/660429502","state":["exists","up"]},{"osd":6,"uuid":"7f4ed35f-4f7c-4538-91b3-d10c93d6ddbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6819","nonce":962000049}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6821","nonce":962000049}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6825","nonce":962000049}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":962000049},{"type":"v1","addr":"172.21.15.103:6823","nonce":962000049}]},"public_addr":"172.21.15.103:6819/962000049","cluster_addr":"172.21.15.103:6821/962000049","heartbeat_back_addr":"172.21.15.103:6825/962000049","heartbeat_front_addr":"172.21.15.103:6823/962000049","state":["exists","up"]},{"osd":7,"uuid":"504cd71d-934e-4a6e-9ed2-5fc4a758800e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6817","nonce":2935555111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6819","nonce":2935555111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6823","nonce":2935555111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":2935555111},{"type":"v1","addr":"172.21.15.117:6821","nonce":2935555111}]},"public_addr":"172.21.15.117:6817/2935555111","cluster_addr":"172.21.15.117:6819/2935555111","heartbeat_back_addr":"172.21.15.117:6823/2935555111","heartbeat_front_addr":"172.21.15.117:6821/2935555111","state":["exists","up"]},{"osd":8,"uuid":"0ede29a2-c541-4834-b320-cd6f66b7846a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6817","nonce":3906862108}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6819","nonce":3906862108}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6823","nonce":3906862108}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3906862108},{"type":"v1","addr":"172.21.15.183:6821","nonce":3906862108}]},"public_addr":"172.21.15.183:6817/3906862108","cluster_addr":"172.21.15.183:6819/3906862108","heartbeat_back_addr":"172.21.15.183:6823/3906862108","heartbeat_front_addr":"172.21.15.183:6821/3906862108","state":["exists","up"]},{"osd":9,"uuid":"7999315b-3589-4be7-9d06-4436f21e3235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6827","nonce":64454554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6829","nonce":64454554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6833","nonce":64454554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":64454554},{"type":"v1","addr":"172.21.15.103:6831","nonce":64454554}]},"public_addr":"172.21.15.103:6827/64454554","cluster_addr":"172.21.15.103:6829/64454554","heartbeat_back_addr":"172.21.15.103:6833/64454554","heartbeat_front_addr":"172.21.15.103:6831/64454554","state":["exists","up"]},{"osd":10,"uuid":"bde15347-8049-49ab-a4e7-06a5cba2d225","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6825","nonce":2500948230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6827","nonce":2500948230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6831","nonce":2500948230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":2500948230},{"type":"v1","addr":"172.21.15.117:6829","nonce":2500948230}]},"public_addr":"172.21.15.117:6825/2500948230","cluster_addr":"172.21.15.117:6827/2500948230","heartbeat_back_addr":"172.21.15.117:6831/2500948230","heartbeat_front_addr":"172.21.15.117:6829/2500948230","state":["exists","up"]},{"osd":11,"uuid":"1bdface1-a9b3-4175-93ea-db2997b3c55b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6825","nonce":405835574}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6827","nonce":405835574}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6831","nonce":405835574}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":405835574},{"type":"v1","addr":"172.21.15.183:6829","nonce":405835574}]},"public_addr":"172.21.15.183:6825/405835574","cluster_addr":"172.21.15.183:6827/405835574","heartbeat_back_addr":"172.21.15.183:6831/405835574","heartbeat_front_addr":"172.21.15.183:6829/405835574","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:28.306577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:14:54.652525+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:19.381282+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:15:50.518766+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:16.653853+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:16:42.479452+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:12.966811+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:17:40.441666+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:05.611234+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:18:36.068690+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:01.974799+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T00:19:27.085865+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.e","mappings":[{"from":5,"to":8}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:0/489365644":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1436162684":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6801/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/817530946":"2024-09-12T00:12:26.257361+0000","172.21.15.103:6800/2651148903":"2024-09-12T00:12:26.257361+0000","172.21.15.103:0/1510088401":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6801/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:6800/477475422":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/3506454988":"2024-09-12T00:11:57.620215+0000","172.21.15.103:0/2690135380":"2024-09-12T00:11:57.620215+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T00:33:36.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2180b5360 msgr2=0x7fe2180b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:36.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2180b5360 0x7fe2180b7820 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe23410f7c0 tx=0x7fe21c005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:36.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe234103c90 msgr2=0x7fe23419b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:36.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe234103c90 0x7fe23419b4b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe224005940 tx=0x7fe224004270 comp rx=0 tx=0).stop 2024-09-11T00:33:36.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 shutdown_connections 2024-09-11T00:33:36.237 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fe2180b5360 0x7fe2180b7820 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe23410d0f0 0x7fe2341a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe2341045b0 0x7fe23419b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 --2- 172.21.15.103:0/636126424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe234103c90 0x7fe23419b4b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:36.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 >> 172.21.15.103:0/636126424 conn(0x7fe2340fd710 msgr2=0x7fe2340fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:36.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 shutdown_connections 2024-09-11T00:33:36.238 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:36.235+0000 7fe239e00640 1 -- 172.21.15.103:0/636126424 wait complete. 2024-09-11T00:33:36.580 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:36 smithi103 ceph-mon[30578]: pgmap v716: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T00:33:36.580 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:36 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/636126424' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:33:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:36 smithi117 ceph-mon[35525]: pgmap v716: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T00:33:36.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:36 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/636126424' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:33:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:36 smithi183 ceph-mon[35934]: pgmap v716: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T00:33:36.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:36 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/636126424' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-11T00:33:37.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T00:33:37.458 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.073+0000 7f86fb200640 1 -- 172.21.15.103:0/2058592146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc10b1a0 msgr2=0x7f86fc1119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.073+0000 7f86fb200640 1 --2- 172.21.15.103:0/2058592146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc10b1a0 0x7f86fc1119a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009ca0 tx=0x7f86f402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.073+0000 7f86fb200640 1 -- 172.21.15.103:0/2058592146 shutdown_connections 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.073+0000 7f86fb200640 1 --2- 172.21.15.103:0/2058592146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc10b1a0 0x7f86fc1119a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.073+0000 7f86fb200640 1 --2- 172.21.15.103:0/2058592146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc10a820 0x7f86fc10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.073+0000 7f86fb200640 1 --2- 172.21.15.103:0/2058592146 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86fc104e10 0x7f86fc1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.073+0000 7f86fb200640 1 -- 172.21.15.103:0/2058592146 >> 172.21.15.103:0/2058592146 conn(0x7f86fc100840 msgr2=0x7f86fc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:38.075 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.074+0000 7f86fb200640 1 -- 172.21.15.103:0/2058592146 shutdown_connections 2024-09-11T00:33:38.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.074+0000 7f86fb200640 1 -- 172.21.15.103:0/2058592146 wait complete. 2024-09-11T00:33:38.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 Processor -- start 2024-09-11T00:33:38.076 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 -- start start 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc104e10 0x7f86fc19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86fc10a820 0x7f86fc19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc10b1a0 0x7f86fc1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc114e60 con 0x7f86fc104e10 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc114ce0 con 0x7f86fc10a820 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fb200640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc114fe0 con 0x7f86fc10b1a0 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86f9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc104e10 0x7f86fc19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86f9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc104e10 0x7f86fc19b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53182/0 (socket says 172.21.15.103:53182) 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86f9e00640 1 -- 172.21.15.103:0/3081287574 learned_addr learned my addr 172.21.15.103:0/3081287574 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86f9400640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86fc10a820 0x7f86fc19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:38.077 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.075+0000 7f86fa800640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc10b1a0 0x7f86fc1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:38.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f9e00640 1 -- 172.21.15.103:0/3081287574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc10b1a0 msgr2=0x7f86fc1a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:38.078 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f9e00640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc10b1a0 0x7f86fc1a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f9e00640 1 -- 172.21.15.103:0/3081287574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86fc10a820 msgr2=0x7f86fc19b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f9e00640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86fc10a820 0x7f86fc19b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f9e00640 1 -- 172.21.15.103:0/3081287574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc1a12c0 con 0x7f86fc104e10 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f9400640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86fc10a820 0x7f86fc19b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f9e00640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc104e10 0x7f86fc19b210 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e9f0 tx=0x7f86ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f2a00640 1 -- 172.21.15.103:0/3081287574 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec00cd80 con 0x7f86fc104e10 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f2a00640 1 -- 172.21.15.103:0/3081287574 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86ec0108e0 con 0x7f86fc104e10 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.076+0000 7f86f2a00640 1 -- 172.21.15.103:0/3081287574 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec018980 con 0x7f86fc104e10 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.077+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc1a15b0 con 0x7f86fc104e10 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.077+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a7c70 con 0x7f86fc104e10 2024-09-11T00:33:38.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.078+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86fc1051f0 con 0x7f86fc104e10 2024-09-11T00:33:38.081 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.080+0000 7f86f2a00640 1 -- 172.21.15.103:0/3081287574 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f86ec010430 con 0x7f86fc104e10 2024-09-11T00:33:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.081+0000 7f86f2a00640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86e00b5410 0x7f86e00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.082+0000 7f86f2a00640 1 -- 172.21.15.103:0/3081287574 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f86ec0e6710 con 0x7f86fc104e10 2024-09-11T00:33:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.082+0000 7f86f9400640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86e00b5410 0x7f86e00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:38.083 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.082+0000 7f86f9400640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86e00b5410 0x7f86e00b78d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f86fc10a3c0 tx=0x7f86e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:38.084 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.083+0000 7f86f2a00640 1 -- 172.21.15.103:0/3081287574 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f86ec0acb90 con 0x7f86fc104e10 2024-09-11T00:33:38.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.303+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f86fc1a8000 con 0x7f86fc104e10 2024-09-11T00:33:38.308 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.306+0000 7f86f2a00640 1 -- 172.21.15.103:0/3081287574 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f86ec0b15c0 con 0x7f86fc104e10 2024-09-11T00:33:38.308 INFO:teuthology.orchestra.run.smithi103.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T00:33:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.310+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86e00b5410 msgr2=0x7f86e00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.310+0000 7f86fb200640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86e00b5410 0x7f86e00b78d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f86fc10a3c0 tx=0x7f86e8005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.310+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc104e10 msgr2=0x7f86fc19b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:38.313 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.310+0000 7f86fb200640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc104e10 0x7f86fc19b210 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00e9f0 tx=0x7f86ec00eeb0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 shutdown_connections 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f86e00b5410 0x7f86e00b78d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f86fc10b1a0 0x7f86fc1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86fc10a820 0x7f86fc19b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 --2- 172.21.15.103:0/3081287574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86fc104e10 0x7f86fc19b210 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 >> 172.21.15.103:0/3081287574 conn(0x7f86fc100840 msgr2=0x7f86fc102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 shutdown_connections 2024-09-11T00:33:38.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:38.312+0000 7f86fb200640 1 -- 172.21.15.103:0/3081287574 wait complete. 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a[30573]: 2024-09-11T00:33:38.304+0000 7f4a06600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-mon[30578]: pgmap v717: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3081287574' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-mon[30578]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-mon[30578]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/3081287574' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-mon[30578]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T00:33:38.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:38 smithi103 ceph-mon[30578]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-11T00:33:38.687 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: ignoring --setuser ceph since I am not root 2024-09-11T00:33:38.687 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: ignoring --setgroup ceph since I am not root 2024-09-11T00:33:38.687 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mds[65297]: -- 172.21.15.117:0/4245479460 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5560c76adc20 con 0x5560c768b400 2024-09-11T00:33:38.688 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: starting mds.b at 2024-09-11T00:33:38.688 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e[64680]: ignoring --setuser ceph since I am not root 2024-09-11T00:33:38.688 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e[64680]: ignoring --setgroup ceph since I am not root 2024-09-11T00:33:38.688 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mds[64684]: -- 172.21.15.117:0/526353742 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55690babd860 con 0x55690ba9b000 2024-09-11T00:33:38.688 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e[64680]: starting mds.e at 2024-09-11T00:33:38.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mon[35525]: pgmap v717: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T00:33:38.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3081287574' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mon[35525]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mon[35525]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/3081287574' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mon[35525]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mon[35525]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h[64980]: ignoring --setuser ceph since I am not root 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h[64980]: ignoring --setgroup ceph since I am not root 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-mds[64984]: -- 172.21.15.117:0/3957574825 <== mon.2 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4661f3c20 con 0x55a4661d1400 2024-09-11T00:33:38.689 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:33:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h[64980]: starting mds.h at 2024-09-11T00:33:38.781 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setuser ceph since I am not root 2024-09-11T00:33:38.781 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: ignoring --setgroup ceph since I am not root 2024-09-11T00:33:38.781 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: starting mds.f at 2024-09-11T00:33:38.781 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c[65748]: ignoring --setuser ceph since I am not root 2024-09-11T00:33:38.781 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c[65748]: ignoring --setgroup ceph since I am not root 2024-09-11T00:33:38.781 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mds[65752]: -- 172.21.15.183:0/3866810376 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633e167b860 con 0x5633e1658800 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c[65748]: starting mds.c at 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i[66053]: ignoring --setuser ceph since I am not root 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i[66053]: ignoring --setgroup ceph since I am not root 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mds[66057]: -- 172.21.15.183:0/3141637815 <== mon.1 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56232d4c7860 con 0x56232d4a4800 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i[66053]: starting mds.i at 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mon[35934]: pgmap v717: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3081287574' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mon[35934]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T00:33:38.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mon[35934]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T00:33:38.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/3081287574' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T00:33:38.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mon[35934]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T00:33:38.783 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:38 smithi183 ceph-mon[35934]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-11T00:33:39.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T00:33:39.778 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:40.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.390+0000 7fcc06800640 1 -- 172.21.15.103:0/2860736895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 msgr2=0x7fcc00077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:40.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.390+0000 7fcc06800640 1 --2- 172.21.15.103:0/2860736895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc00077ab0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009ca0 tx=0x7fcbf402a8c0 comp rx=0 tx=0).stop 2024-09-11T00:33:40.432 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.391+0000 7fcc06800640 1 -- 172.21.15.103:0/2860736895 shutdown_connections 2024-09-11T00:33:40.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.391+0000 7fcc06800640 1 --2- 172.21.15.103:0/2860736895 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc001137e0 0x7fcc00115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:40.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.391+0000 7fcc06800640 1 --2- 172.21.15.103:0/2860736895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc00077ff0 0x7fcc001132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:40.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.391+0000 7fcc06800640 1 --2- 172.21.15.103:0/2860736895 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc00077ab0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:40.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.391+0000 7fcc06800640 1 -- 172.21.15.103:0/2860736895 >> 172.21.15.103:0/2860736895 conn(0x7fcc00100840 msgr2=0x7fcc00102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:40.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.391+0000 7fcc06800640 1 -- 172.21.15.103:0/2860736895 shutdown_connections 2024-09-11T00:33:40.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.391+0000 7fcc06800640 1 -- 172.21.15.103:0/2860736895 wait complete. 2024-09-11T00:33:40.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 Processor -- start 2024-09-11T00:33:40.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 -- start start 2024-09-11T00:33:40.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc0019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:40.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc00077ff0 0x7fcc0019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:40.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc001137e0 0x7fcc001a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:40.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcc001191f0 con 0x7fcc000776d0 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00119070 con 0x7fcc00077ff0 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc06800640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcc00119370 con 0x7fcc001137e0 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc05400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc0019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc05e00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc001137e0 0x7fcc001a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc05400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc0019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:53196/0 (socket says 172.21.15.103:53196) 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc05400640 1 -- 172.21.15.103:0/1118212462 learned_addr learned my addr 172.21.15.103:0/1118212462 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.392+0000 7fcc04a00640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc00077ff0 0x7fcc0019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:40.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc05e00640 1 -- 172.21.15.103:0/1118212462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc00077ff0 msgr2=0x7fcc0019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc05e00640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc00077ff0 0x7fcc0019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc05e00640 1 -- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 msgr2=0x7fcc0019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc05e00640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc0019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc05e00640 1 -- 172.21.15.103:0/1118212462 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc001a56b0 con 0x7fcc001137e0 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc05400640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc0019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc05e00640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc001137e0 0x7fcc001a4ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d650 tx=0x7fcbf800db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcbfe000640 1 -- 172.21.15.103:0/1118212462 <== mon.1 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8017070 con 0x7fcc001137e0 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcbfe000640 1 -- 172.21.15.103:0/1118212462 <== mon.1 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf8009820 con 0x7fcc001137e0 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcbfe000640 1 -- 172.21.15.103:0/1118212462 <== mon.1 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8004c60 con 0x7fcc001137e0 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.393+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc001a59a0 con 0x7fcc001137e0 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.394+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc001ac060 con 0x7fcc001137e0 2024-09-11T00:33:40.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.395+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc00077ab0 con 0x7fcc001137e0 2024-09-11T00:33:40.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.397+0000 7fcbfe000640 1 -- 172.21.15.103:0/1118212462 <== mon.1 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7fcbf8049aa0 con 0x7fcc001137e0 2024-09-11T00:33:40.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.398+0000 7fcbfe000640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcbe80b5390 0x7fcbe80b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:40.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.398+0000 7fcbfe000640 1 -- 172.21.15.103:0/1118212462 <== mon.1 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7fcbf80e5970 con 0x7fcc001137e0 2024-09-11T00:33:40.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.398+0000 7fcc05400640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcbe80b5390 0x7fcbe80b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:40.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.399+0000 7fcc05400640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcbe80b5390 0x7fcbe80b7850 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4004ad0 tx=0x7fcbf4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:40.437 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.400+0000 7fcbfe000640 1 -- 172.21.15.103:0/1118212462 <== mon.1 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcbf80abd10 con 0x7fcc001137e0 2024-09-11T00:33:40.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: pgmap v719: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-11T00:33:40.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6834/2746286829,v1:172.21.15.117:6835/2746286829] up:boot 2024-09-11T00:33:40.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6834/1689455359,v1:172.21.15.183:6835/1689455359] up:boot 2024-09-11T00:33:40.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6836/3547531842,v1:172.21.15.183:6838/3547531842] up:boot 2024-09-11T00:33:40.581 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.183:6837/1627280681,v1:172.21.15.183:6839/1627280681] up:boot 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6836/2614994493,v1:172.21.15.117:6837/2614994493] up:boot 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: mds.? [v2:172.21.15.117:6838/1709410450,v1:172.21.15.117:6839/1709410450] up:boot 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:33:40.582 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:40 smithi103 ceph-mon[30578]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:33:40.625 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:40.624+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc001ac320 con 0x7fcc001137e0 2024-09-11T00:33:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: pgmap v719: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-11T00:33:40.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6834/2746286829,v1:172.21.15.117:6835/2746286829] up:boot 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6834/1689455359,v1:172.21.15.183:6835/1689455359] up:boot 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6836/3547531842,v1:172.21.15.183:6838/3547531842] up:boot 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.183:6837/1627280681,v1:172.21.15.183:6839/1627280681] up:boot 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6836/2614994493,v1:172.21.15.117:6837/2614994493] up:boot 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: mds.? [v2:172.21.15.117:6838/1709410450,v1:172.21.15.117:6839/1709410450] up:boot 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:33:40.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:40 smithi117 ceph-mon[35525]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: pgmap v719: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6834/2746286829,v1:172.21.15.117:6835/2746286829] up:boot 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6834/1689455359,v1:172.21.15.183:6835/1689455359] up:boot 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6836/3547531842,v1:172.21.15.183:6838/3547531842] up:boot 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.183:6837/1627280681,v1:172.21.15.183:6839/1627280681] up:boot 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6836/2614994493,v1:172.21.15.117:6837/2614994493] up:boot 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: mds.? [v2:172.21.15.117:6838/1709410450,v1:172.21.15.117:6839/1709410450] up:boot 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-11T00:33:40.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-11T00:33:40.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-11T00:33:40.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-11T00:33:40.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-11T00:33:40.782 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:40 smithi183 ceph-mon[35934]: from='mgr.14150 172.21.15.103:0/3479578479' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-11T00:33:41.350 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.348+0000 7fcbfe000640 1 -- 172.21.15.103:0/1118212462 <== mon.1 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7fcbf80b0740 con 0x7fcc001137e0 2024-09-11T00:33:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcbe80b5390 msgr2=0x7fcbe80b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:41.352 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcbe80b5390 0x7fcbe80b7850 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4004ad0 tx=0x7fcbf4004a40 comp rx=0 tx=0).stop 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc001137e0 msgr2=0x7fcc001a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc001137e0 0x7fcc001a4ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800d650 tx=0x7fcbf800db10 comp rx=0 tx=0).stop 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 shutdown_connections 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7fcbe80b5390 0x7fcbe80b7850 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcc001137e0 0x7fcc001a4ee0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc00077ff0 0x7fcc0019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 --2- 172.21.15.103:0/1118212462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcc000776d0 0x7fcc0019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 >> 172.21.15.103:0/1118212462 conn(0x7fcc00100840 msgr2=0x7fcc00102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 shutdown_connections 2024-09-11T00:33:41.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:41.351+0000 7fcc06800640 1 -- 172.21.15.103:0/1118212462 wait complete. 2024-09-11T00:33:41.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:41 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1118212462' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:41.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:41 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:41 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1118212462' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:41.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:41 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:41 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1118212462' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:41.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:41 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-11T00:33:42.141 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T00:33:42.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:42 smithi103 ceph-mon[30578]: pgmap v720: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-11T00:33:42.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:42 smithi103 ceph-mon[30578]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T00:33:42.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:42 smithi103 ceph-mon[30578]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T00:33:42.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:42 smithi103 ceph-mon[30578]: Cluster is now healthy 2024-09-11T00:33:42.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:42 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:33:42.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:42 smithi103 ceph-mon[30578]: fsmap 12 up:standby 2024-09-11T00:33:42.524 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:42 smithi117 ceph-mon[35525]: pgmap v720: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-11T00:33:42.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:42 smithi117 ceph-mon[35525]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T00:33:42.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:42 smithi117 ceph-mon[35525]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T00:33:42.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:42 smithi117 ceph-mon[35525]: Cluster is now healthy 2024-09-11T00:33:42.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:42 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:33:42.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:42 smithi117 ceph-mon[35525]: fsmap 12 up:standby 2024-09-11T00:33:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:42 smithi183 ceph-mon[35934]: pgmap v720: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-11T00:33:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:42 smithi183 ceph-mon[35934]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T00:33:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:42 smithi183 ceph-mon[35934]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T00:33:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:42 smithi183 ceph-mon[35934]: Cluster is now healthy 2024-09-11T00:33:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:42 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T00:33:42.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:42 smithi183 ceph-mon[35934]: fsmap 12 up:standby 2024-09-11T00:33:43.102 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 -- 172.21.15.103:0/899893871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 msgr2=0x7f22ec108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 --2- 172.21.15.103:0/899893871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec108a50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009900 tx=0x7f22d802a410 comp rx=0 tx=0).stop 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 -- 172.21.15.103:0/899893871 shutdown_connections 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 --2- 172.21.15.103:0/899893871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec108f90 0x7f22ec10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 --2- 172.21.15.103:0/899893871 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec108a50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 --2- 172.21.15.103:0/899893871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec102c00 0x7f22ec102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 -- 172.21.15.103:0/899893871 >> 172.21.15.103:0/899893871 conn(0x7f22ec0fe6d0 msgr2=0x7f22ec100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 -- 172.21.15.103:0/899893871 shutdown_connections 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.101+0000 7f22ebe00640 1 -- 172.21.15.103:0/899893871 wait complete. 2024-09-11T00:33:43.103 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.102+0000 7f22ebe00640 1 Processor -- start 2024-09-11T00:33:43.104 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ebe00640 1 -- start start 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ebe00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec102c00 0x7f22ec19b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ebe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec19b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ebe00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec108f90 0x7f22ec1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ebe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112c50 con 0x7f22ec108610 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ebe00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112ad0 con 0x7f22ec102c00 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ebe00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f22ec112dd0 con 0x7f22ec108f90 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.103+0000 7f22ea000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec19b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22eaa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec102c00 0x7f22ec19b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec19b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36126/0 (socket says 172.21.15.103:36126) 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 -- 172.21.15.103:0/339456304 learned_addr learned my addr 172.21.15.103:0/339456304 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:43.105 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22eb400640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec108f90 0x7f22ec1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:43.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 -- 172.21.15.103:0/339456304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec108f90 msgr2=0x7f22ec1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:43.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec108f90 0x7f22ec1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 -- 172.21.15.103:0/339456304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec102c00 msgr2=0x7f22ec19b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:43.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec102c00 0x7f22ec19b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 -- 172.21.15.103:0/339456304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22ec1a1380 con 0x7f22ec108610 2024-09-11T00:33:43.106 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22eaa00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec102c00 0x7f22ec19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:43.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.104+0000 7f22ea000640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec19b810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005940 tx=0x7f22d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:43.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.105+0000 7f22e3400640 1 -- 172.21.15.103:0/339456304 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d803f030 con 0x7f22ec108610 2024-09-11T00:33:43.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.105+0000 7f22e3400640 1 -- 172.21.15.103:0/339456304 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d8003aa0 con 0x7f22ec108610 2024-09-11T00:33:43.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.105+0000 7f22e3400640 1 -- 172.21.15.103:0/339456304 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d8035070 con 0x7f22ec108610 2024-09-11T00:33:43.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.105+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22ec1a1610 con 0x7f22ec108610 2024-09-11T00:33:43.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.105+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22ec1a7d30 con 0x7f22ec108610 2024-09-11T00:33:43.107 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.106+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22ec102fe0 con 0x7f22ec108610 2024-09-11T00:33:43.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.108+0000 7f22e3400640 1 -- 172.21.15.103:0/339456304 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f22d803bce0 con 0x7f22ec108610 2024-09-11T00:33:43.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.110+0000 7f22e3400640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f22d00b5410 0x7f22d00b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:43.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.110+0000 7f22e3400640 1 -- 172.21.15.103:0/339456304 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9408+0+0 (secure 0 0 0) 0x7f22d8106990 con 0x7f22ec108610 2024-09-11T00:33:43.111 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.110+0000 7f22eaa00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f22d00b5410 0x7f22d00b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:43.112 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.111+0000 7f22eaa00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f22d00b5410 0x7f22d00b78d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009fc0 tx=0x7f22dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:43.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.116+0000 7f22e3400640 1 -- 172.21.15.103:0/339456304 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f22d80ccd30 con 0x7f22ec108610 2024-09-11T00:33:43.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.304+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22ec02e5c0 con 0x7f22ec108610 2024-09-11T00:33:43.349 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.348+0000 7f22e3400640 1 -- 172.21.15.103:0/339456304 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f22d80d1760 con 0x7f22ec108610 2024-09-11T00:33:43.350 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_metadata' removed 2024-09-11T00:33:43.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f22d00b5410 msgr2=0x7f22d00b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:43.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f22d00b5410 0x7f22d00b78d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009fc0 tx=0x7f22dc005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:43.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 msgr2=0x7f22ec19b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:43.353 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec19b810 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f22d8005940 tx=0x7f22d8004270 comp rx=0 tx=0).stop 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 shutdown_connections 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f22d00b5410 0x7f22d00b78d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f22ec108f90 0x7f22ec1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f22ec108610 0x7f22ec19b810 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 --2- 172.21.15.103:0/339456304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec102c00 0x7f22ec19b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.352+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 >> 172.21.15.103:0/339456304 conn(0x7f22ec0fe6d0 msgr2=0x7f22ec105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.353+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 shutdown_connections 2024-09-11T00:33:43.354 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:43.353+0000 7f22ebe00640 1 -- 172.21.15.103:0/339456304 wait complete. 2024-09-11T00:33:43.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:43 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/339456304' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:43.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:43 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/339456304' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:43.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:43 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/339456304' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:44.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T00:33:44.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:44 smithi103 ceph-mon[30578]: pgmap v721: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-11T00:33:44.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:44 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/339456304' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:44.671 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:44 smithi103 ceph-mon[30578]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T00:33:44.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:44 smithi117 ceph-mon[35525]: pgmap v721: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-11T00:33:44.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:44 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/339456304' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:44.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:44 smithi117 ceph-mon[35525]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T00:33:44.711 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:44.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:44 smithi183 ceph-mon[35934]: pgmap v721: 129 pgs: 129 active+clean; 44 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 614 B/s wr, 6 op/s 2024-09-11T00:33:44.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:44 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/339456304' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:44.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:44 smithi183 ceph-mon[35934]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 -- 172.21.15.103:0/712800833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 msgr2=0x7f8a08077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/712800833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a08077ab0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009900 tx=0x7f89fc02a420 comp rx=0 tx=0).stop 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 -- 172.21.15.103:0/712800833 shutdown_connections 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/712800833 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08113990 0x7f8a08115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/712800833 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08077ff0 0x7f8a08113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/712800833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a08077ab0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 -- 172.21.15.103:0/712800833 >> 172.21.15.103:0/712800833 conn(0x7f8a08100a50 msgr2=0x7f8a08102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 -- 172.21.15.103:0/712800833 shutdown_connections 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.304+0000 7f8a0fa00640 1 -- 172.21.15.103:0/712800833 wait complete. 2024-09-11T00:33:45.340 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 Processor -- start 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 -- start start 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a0819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08077ff0 0x7f8a0819ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08113990 0x7f8a081a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a081193f0 con 0x7f8a080776d0 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08119270 con 0x7f8a08077ff0 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0fa00640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a08119570 con 0x7f8a08113990 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.306+0000 7f8a0d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a0819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a0819f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:36132/0 (socket says 172.21.15.103:36132) 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 -- 172.21.15.103:0/1221150322 learned_addr learned my addr 172.21.15.103:0/1221150322 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a07e00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08077ff0 0x7f8a0819ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:45.341 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 -- 172.21.15.103:0/1221150322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08113990 msgr2=0x7f8a081a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08113990 0x7f8a081a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 -- 172.21.15.103:0/1221150322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08077ff0 msgr2=0x7f8a0819ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08077ff0 0x7f8a0819ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 -- 172.21.15.103:0/1221150322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a081a5a70 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a07e00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08077ff0 0x7f8a0819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.307+0000 7f8a0d000640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a0819f9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89fc005940 tx=0x7f89fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.308+0000 7f8a05600640 1 -- 172.21.15.103:0/1221150322 <== mon.0 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc03f030 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.308+0000 7f8a05600640 1 -- 172.21.15.103:0/1221150322 <== mon.0 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89fc003cb0 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.308+0000 7f8a05600640 1 -- 172.21.15.103:0/1221150322 <== mon.0 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc035070 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.308+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a081a5d00 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.308+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a081ac420 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.309+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a08077ab0 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.311+0000 7f8a05600640 1 -- 172.21.15.103:0/1221150322 <== mon.0 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f89fc003800 con 0x7f8a080776d0 2024-09-11T00:33:45.342 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.313+0000 7f8a05600640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89f00b53e0 0x7f89f00b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.313+0000 7f8a05600640 1 -- 172.21.15.103:0/1221150322 <== mon.0 v2:172.21.15.103:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8912+0+0 (secure 0 0 0) 0x7f89fc1068f0 con 0x7f8a080776d0 2024-09-11T00:33:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.313+0000 7f8a07e00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89f00b53e0 0x7f89f00b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.314+0000 7f8a07e00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89f00b53e0 0x7f89f00b78a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f89f40095f0 tx=0x7f89f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:45.343 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.316+0000 7f8a05600640 1 -- 172.21.15.103:0/1221150322 <== mon.0 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f89fc0cd4d0 con 0x7f8a080776d0 2024-09-11T00:33:45.510 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:45.508+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8a0802e5c0 con 0x7f8a080776d0 2024-09-11T00:33:46.362 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.358+0000 7f8a05600640 1 -- 172.21.15.103:0/1221150322 <== mon.0 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f89fc0ccdd0 con 0x7f8a080776d0 2024-09-11T00:33:46.362 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data' removed 2024-09-11T00:33:46.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.361+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89f00b53e0 msgr2=0x7f89f00b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:46.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.361+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89f00b53e0 0x7f89f00b78a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f89f40095f0 tx=0x7f89f4005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:46.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.361+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 msgr2=0x7f8a0819f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:46.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.361+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a0819f9c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f89fc005940 tx=0x7f89fc0042f0 comp rx=0 tx=0).stop 2024-09-11T00:33:46.364 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.362+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 shutdown_connections 2024-09-11T00:33:46.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.362+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f89f00b53e0 0x7f89f00b78a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:46.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.362+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a08113990 0x7f8a081a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:46.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.362+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a08077ff0 0x7f8a0819ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:46.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.362+0000 7f8a0fa00640 1 --2- 172.21.15.103:0/1221150322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a080776d0 0x7f8a0819f9c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:46.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.362+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 >> 172.21.15.103:0/1221150322 conn(0x7f8a08100a50 msgr2=0x7f8a08114590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:46.365 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.363+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 shutdown_connections 2024-09-11T00:33:46.366 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:46.363+0000 7f8a0fa00640 1 -- 172.21.15.103:0/1221150322 wait complete. 2024-09-11T00:33:46.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:46 smithi103 ceph-mon[30578]: pgmap v723: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:46.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:46 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1221150322' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:46 smithi117 ceph-mon[35525]: pgmap v723: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:46.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:46 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1221150322' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:46.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:46 smithi183 ceph-mon[35934]: pgmap v723: 97 pgs: 97 active+clean; 577 KiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:46.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:46 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1221150322' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:47.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-11T00:33:47.639 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T00:33:47.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:47 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/1221150322' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:47.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:47 smithi103 ceph-mon[30578]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T00:33:47.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:47 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/1221150322' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:47.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:47 smithi117 ceph-mon[35525]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T00:33:47.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:47 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/1221150322' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:47.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:47 smithi183 ceph-mon[35934]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 -- 172.21.15.103:0/2655061078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105ef0 msgr2=0x7f940c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 --2- 172.21.15.103:0/2655061078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105ef0 0x7f940c10f930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9404009ca0 tx=0x7f940402a8b0 comp rx=0 tx=0).stop 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 -- 172.21.15.103:0/2655061078 shutdown_connections 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 --2- 172.21.15.103:0/2655061078 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105ef0 0x7f940c10f930 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 --2- 172.21.15.103:0/2655061078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f940c105570 0x7f940c1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 --2- 172.21.15.103:0/2655061078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c101220 0x7f940c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 -- 172.21.15.103:0/2655061078 >> 172.21.15.103:0/2655061078 conn(0x7f940c078100 msgr2=0x7f940c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 -- 172.21.15.103:0/2655061078 shutdown_connections 2024-09-11T00:33:48.258 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.255+0000 7f9412600640 1 -- 172.21.15.103:0/2655061078 wait complete. 2024-09-11T00:33:48.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.256+0000 7f9412600640 1 Processor -- start 2024-09-11T00:33:48.259 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.256+0000 7f9412600640 1 -- start start 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f9412600640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f940c101220 0x7f940c1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f9412600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105570 0x7f940c102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f9412600640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c105ef0 0x7f940c102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f9412600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f940c112db0 con 0x7f940c105570 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f9412600640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 -- 0x7f940c112c30 con 0x7f940c105ef0 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f9412600640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f940c112f30 con 0x7f940c101220 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f940be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c105ef0 0x7f940c102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f940be00640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c105ef0 0x7f940c102c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.103:43366/0 (socket says 172.21.15.103:43366) 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f9403e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105570 0x7f940c102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f940be00640 1 -- 172.21.15.103:0/2337179974 learned_addr learned my addr 172.21.15.103:0/2337179974 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-09-11T00:33:48.260 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.257+0000 7f940b400640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f940c101220 0x7f940c1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:48.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f940be00640 1 -- 172.21.15.103:0/2337179974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f940c101220 msgr2=0x7f940c1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:48.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f940be00640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f940c101220 0x7f940c1021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:48.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f940be00640 1 -- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105570 msgr2=0x7f940c102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:48.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f940be00640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105570 0x7f940c102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:48.261 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f940be00640 1 -- 172.21.15.103:0/2337179974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c1a7680 con 0x7f940c105ef0 2024-09-11T00:33:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f940be00640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c105ef0 0x7f940c102c70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9404005cc0 tx=0x7f9404030db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f9408c00640 1 -- 172.21.15.103:0/2337179974 <== mon.2 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404003e00 con 0x7f940c105ef0 2024-09-11T00:33:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f9408c00640 1 -- 172.21.15.103:0/2337179974 <== mon.2 v2:172.21.15.117:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9404040070 con 0x7f940c105ef0 2024-09-11T00:33:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940c1a7910 con 0x7f940c105ef0 2024-09-11T00:33:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.258+0000 7f9408c00640 1 -- 172.21.15.103:0/2337179974 <== mon.2 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404045a60 con 0x7f940c105ef0 2024-09-11T00:33:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.259+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940c1a7ea0 con 0x7f940c105ef0 2024-09-11T00:33:48.262 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.260+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f940c101600 con 0x7f940c105ef0 2024-09-11T00:33:48.263 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.262+0000 7f9408c00640 1 -- 172.21.15.103:0/2337179974 <== mon.2 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 18) ==== 153251+0+0 (secure 0 0 0) 0x7f940404d030 con 0x7f940c105ef0 2024-09-11T00:33:48.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.263+0000 7f9408c00640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f93f00b5360 0x7f93f00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T00:33:48.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.263+0000 7f9408c00640 1 -- 172.21.15.103:0/2337179974 <== mon.2 v2:172.21.15.117:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8324+0+0 (secure 0 0 0) 0x7f9404106960 con 0x7f940c105ef0 2024-09-11T00:33:48.265 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.263+0000 7f940b400640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f93f00b5360 0x7f93f00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T00:33:48.266 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.264+0000 7f940b400640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f93f00b5360 0x7f93f00b7820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009bd0 tx=0x7f93fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T00:33:48.267 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.265+0000 7f9408c00640 1 -- 172.21.15.103:0/2337179974 <== mon.2 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f94040cd760 con 0x7f940c105ef0 2024-09-11T00:33:48.419 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:48 smithi103 ceph-mon[30578]: pgmap v725: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:48.459 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:48.457+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f940c1034f0 con 0x7f940c105ef0 2024-09-11T00:33:48.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:48 smithi117 ceph-mon[35525]: pgmap v725: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:48.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:48 smithi183 ceph-mon[35934]: pgmap v725: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:49.382 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.377+0000 7f9408c00640 1 -- 172.21.15.103:0/2337179974 <== mon.2 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v96) ==== 178+0+0 (secure 0 0 0) 0x7f940c1034f0 con 0x7f940c105ef0 2024-09-11T00:33:49.382 INFO:teuthology.orchestra.run.smithi103.stderr:pool 'cephfs_data_ec' removed 2024-09-11T00:33:49.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.381+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f93f00b5360 msgr2=0x7f93f00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:49.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.381+0000 7f9412600640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f93f00b5360 0x7f93f00b7820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009bd0 tx=0x7f93fc005f90 comp rx=0 tx=0).stop 2024-09-11T00:33:49.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.381+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c105ef0 msgr2=0x7f940c102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T00:33:49.383 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.381+0000 7f9412600640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c105ef0 0x7f940c102c70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9404005cc0 tx=0x7f9404030db0 comp rx=0 tx=0).stop 2024-09-11T00:33:49.384 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.382+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 shutdown_connections 2024-09-11T00:33:49.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.382+0000 7f9412600640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:6800/3157485146,v1:172.21.15.103:6801/3157485146] conn(0x7f93f00b5360 0x7f93f00b7820 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:49.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.382+0000 7f9412600640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f940c105ef0 0x7f940c102c70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:49.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.382+0000 7f9412600640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f940c105570 0x7f940c102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:49.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.382+0000 7f9412600640 1 --2- 172.21.15.103:0/2337179974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f940c101220 0x7f940c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T00:33:49.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.382+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 >> 172.21.15.103:0/2337179974 conn(0x7f940c078100 msgr2=0x7f940c1070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T00:33:49.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.383+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 shutdown_connections 2024-09-11T00:33:49.385 INFO:teuthology.orchestra.run.smithi103.stderr:2024-09-11T00:33:49.383+0000 7f9412600640 1 -- 172.21.15.103:0/2337179974 wait complete. 2024-09-11T00:33:49.669 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:49 smithi103 ceph-mon[30578]: from='client.? 172.21.15.103:0/2337179974' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:49.670 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:49 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:49.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:49 smithi117 ceph-mon[35525]: from='client.? 172.21.15.103:0/2337179974' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:49.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:49 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:49.780 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:49 smithi183 ceph-mon[35934]: from='client.? 172.21.15.103:0/2337179974' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:49.781 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:49 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-11T00:33:50.346 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:33:50.379 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:50 smithi183 ceph-mon[35934]: pgmap v726: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:50.379 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:50 smithi183 ceph-mon[35934]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:50.379 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:50 smithi183 ceph-mon[35934]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T00:33:50.408 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:33:50.457 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:33:50.493 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T00:33:50.493 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:33:50.530 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:33:50.546 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T00:33:50.562 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T00:33:50.562 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T00:33:50.562 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a 2024-09-11T00:33:50.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:50 smithi103 ceph-mon[30578]: pgmap v726: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:50.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:50 smithi103 ceph-mon[30578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:50.618 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:50 smithi103 ceph-mon[30578]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T00:33:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:50 smithi117 ceph-mon[35525]: pgmap v726: 33 pgs: 33 active+clean; 577 KiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T00:33:50.687 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:50 smithi117 ceph-mon[35525]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T00:33:50.688 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:50 smithi117 ceph-mon[35525]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T00:33:50.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:50 smithi103 systemd[1]: Stopping Ceph mon.a for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:33:51.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:50 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a[30573]: 2024-09-11T00:33:50.994+0000 7f4a0d400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:51.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:50 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a[30573]: 2024-09-11T00:33:50.994+0000 7f4a0d400640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T00:33:51.420 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:51 smithi103 podman[116177]: 2024-09-11 00:33:51.221308396 +0000 UTC m=+0.364403503 container died 0cabdefb1d1cf21f3b86fee4007caef54c51dd9f0be7da85def4f7a0745cef9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:33:51.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:51 smithi103 podman[116177]: 2024-09-11 00:33:51.72572536 +0000 UTC m=+0.868820465 container cleanup 0cabdefb1d1cf21f3b86fee4007caef54c51dd9f0be7da85def4f7a0745cef9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:33:51.920 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:51 smithi103 bash[116177]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a 2024-09-11T00:33:52.236 INFO:journalctl@ceph.mon.a.smithi103.stdout:Sep 11 00:33:51 smithi103 podman[116190]: 2024-09-11 00:33:51.977450346 +0000 UTC m=+0.742793754 container remove 0cabdefb1d1cf21f3b86fee4007caef54c51dd9f0be7da85def4f7a0745cef9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS) 2024-09-11T00:33:52.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T00:33:52.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:52.338 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T00:33:52.339 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T00:33:52.339 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.b 2024-09-11T00:33:52.677 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:52 smithi117 systemd[1]: Stopping Ceph mon.b for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:33:52.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-b[35521]: 2024-09-11T00:33:52.674+0000 7f2d35f9d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:52.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-b[35521]: 2024-09-11T00:33:52.674+0000 7f2d35f9d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T00:33:52.938 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:52 smithi117 podman[69538]: 2024-09-11 00:33:52.828289025 +0000 UTC m=+0.283910449 container died 180a9c40821124b92ef2589752e0c6619b3ca7443366e4fec722e2e4a41884ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T00:33:53.437 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:53 smithi117 podman[69538]: 2024-09-11 00:33:53.225750284 +0000 UTC m=+0.681371707 container cleanup 180a9c40821124b92ef2589752e0c6619b3ca7443366e4fec722e2e4a41884ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T00:33:53.438 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 11 00:33:53 smithi117 bash[69538]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-b 2024-09-11T00:33:53.606 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T00:33:53.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:53.650 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T00:33:53.651 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T00:33:53.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.c 2024-09-11T00:33:53.994 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:53 smithi183 systemd[1]: Stopping Ceph mon.c for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:33:54.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-c[35930]: 2024-09-11T00:33:53.990+0000 7f14c471b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:54.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-c[35930]: 2024-09-11T00:33:53.990+0000 7f14c471b640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T00:33:54.281 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:54 smithi183 podman[70639]: 2024-09-11 00:33:54.114919388 +0000 UTC m=+0.269505815 container died c3c8548ca275e798a57b47fee52dc03870ce12b86600e3b4f7dd313e5519b98c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:33:54.779 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:54 smithi183 podman[70639]: 2024-09-11 00:33:54.51765702 +0000 UTC m=+0.672243451 container cleanup c3c8548ca275e798a57b47fee52dc03870ce12b86600e3b4f7dd313e5519b98c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T00:33:54.779 INFO:journalctl@ceph.mon.c.smithi183.stdout:Sep 11 00:33:54 smithi183 bash[70639]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mon-c 2024-09-11T00:33:54.944 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T00:33:55.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:55.023 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T00:33:55.023 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T00:33:55.023 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.x 2024-09-11T00:33:55.420 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:33:55 smithi103 systemd[1]: Stopping Ceph mgr.x for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:33:55.918 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:33:55 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:33:55.458+0000 7f7906600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:33:55.918 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:33:55 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x[30801]: 2024-09-11T00:33:55.458+0000 7f7906600640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T00:33:55.918 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:33:55 smithi103 podman[116294]: 2024-09-11 00:33:55.541445665 +0000 UTC m=+0.235909598 container died 5762ba2c7558eb5228a1a46d46dc6d101325ce9cba8622469fb0fb9729e9d2e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:33:56.170 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:33:56 smithi103 podman[116294]: 2024-09-11 00:33:56.098043393 +0000 UTC m=+0.792507324 container cleanup 5762ba2c7558eb5228a1a46d46dc6d101325ce9cba8622469fb0fb9729e9d2e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T00:33:56.170 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:33:56 smithi103 bash[116294]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x 2024-09-11T00:33:56.779 INFO:journalctl@ceph.mgr.x.smithi103.stdout:Sep 11 00:33:56 smithi103 podman[116305]: 2024-09-11 00:33:56.475423876 +0000 UTC m=+0.919450912 container remove 5762ba2c7558eb5228a1a46d46dc6d101325ce9cba8622469fb0fb9729e9d2e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T00:33:56.818 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T00:33:56.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:56.891 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T00:33:56.892 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T00:33:56.892 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.y 2024-09-11T00:33:57.187 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 11 00:33:56 smithi117 systemd[1]: Stopping Ceph mgr.y for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:33:57.641 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 11 00:33:57 smithi117 podman[69664]: 2024-09-11 00:33:57.226433989 +0000 UTC m=+0.142456058 container died c174358fee869e9f7befe1027a9d387c9f705582fb62e780bad566280fc07c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:33:57.937 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 11 00:33:57 smithi117 podman[69664]: 2024-09-11 00:33:57.731381466 +0000 UTC m=+0.647403536 container cleanup c174358fee869e9f7befe1027a9d387c9f705582fb62e780bad566280fc07c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T00:33:57.938 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Sep 11 00:33:57 smithi117 bash[69664]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-y 2024-09-11T00:33:58.321 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T00:33:58.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:58.356 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T00:33:58.356 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T00:33:58.356 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.z 2024-09-11T00:33:58.694 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:33:58 smithi183 systemd[1]: Stopping Ceph mgr.z for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:33:59.000 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:33:58 smithi183 podman[70764]: 2024-09-11 00:33:58.693371226 +0000 UTC m=+0.143431052 container died 81633643c467e3ed145038fd07fac50dacfae5118462b3f9931073d7eb959905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-11T00:33:59.281 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:33:59 smithi183 podman[70764]: 2024-09-11 00:33:59.173349609 +0000 UTC m=+0.623409434 container cleanup 81633643c467e3ed145038fd07fac50dacfae5118462b3f9931073d7eb959905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T00:33:59.281 INFO:journalctl@ceph.mgr.z.smithi183.stdout:Sep 11 00:33:59 smithi183 bash[70764]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mgr-z 2024-09-11T00:33:59.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T00:33:59.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:33:59.631 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T00:33:59.631 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T00:33:59.631 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.0 2024-09-11T00:34:00.058 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:33:59 smithi103 systemd[1]: Stopping Ceph osd.0 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:00.420 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:00 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0[46487]: 2024-09-11T00:34:00.055+0000 7f652b200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:00.420 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:00 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0[46487]: 2024-09-11T00:34:00.055+0000 7f652b200640 -1 osd.0 96 *** Got signal Terminated *** 2024-09-11T00:34:00.420 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:00 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0[46487]: 2024-09-11T00:34:00.055+0000 7f652b200640 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:05.920 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:05 smithi103 podman[116419]: 2024-09-11 00:34:05.569895487 +0000 UTC m=+5.649305179 container died c4e18bd557be87c52f5fba0ee2a41776ac85fae859690dc18bcdc8b3d6dd15f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T00:34:06.358 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:06 smithi103 podman[116419]: 2024-09-11 00:34:06.112765618 +0000 UTC m=+6.192175308 container cleanup c4e18bd557be87c52f5fba0ee2a41776ac85fae859690dc18bcdc8b3d6dd15f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T00:34:06.358 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:06 smithi103 bash[116419]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0 2024-09-11T00:34:06.670 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:06 smithi103 podman[116436]: 2024-09-11 00:34:06.356559535 +0000 UTC m=+0.775799016 container remove c4e18bd557be87c52f5fba0ee2a41776ac85fae859690dc18bcdc8b3d6dd15f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:07.237 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:06 smithi103 podman[116525]: 2024-09-11 00:34:06.834209784 +0000 UTC m=+0.027234962 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:07.238 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:06 smithi103 podman[116525]: 2024-09-11 00:34:06.965752519 +0000 UTC m=+0.158777695 container create fd50112eeac84fe4142c71184f7d6c438aebd2e890c0868a908c59d054130c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T00:34:07.238 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:07 smithi103 podman[116525]: 2024-09-11 00:34:07.235518127 +0000 UTC m=+0.428543306 container init fd50112eeac84fe4142c71184f7d6c438aebd2e890c0868a908c59d054130c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:07.520 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:07 smithi103 podman[116525]: 2024-09-11 00:34:07.243453498 +0000 UTC m=+0.436478684 container start fd50112eeac84fe4142c71184f7d6c438aebd2e890c0868a908c59d054130c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:07.520 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:07 smithi103 podman[116525]: 2024-09-11 00:34:07.343557197 +0000 UTC m=+0.536582393 container attach fd50112eeac84fe4142c71184f7d6c438aebd2e890c0868a908c59d054130c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T00:34:07.520 INFO:journalctl@ceph.osd.0.smithi103.stdout:Sep 11 00:34:07 smithi103 podman[116525]: 2024-09-11 00:34:07.518832167 +0000 UTC m=+0.711857356 container died fd50112eeac84fe4142c71184f7d6c438aebd2e890c0868a908c59d054130c68 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:34:08.314 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T00:34:08.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:08.392 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T00:34:08.392 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T00:34:08.392 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.1 2024-09-11T00:34:08.687 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:08 smithi117 systemd[1]: Stopping Ceph osd.1 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:09.187 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1[40006]: 2024-09-11T00:34:08.727+0000 7f8ac67d2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:09.187 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1[40006]: 2024-09-11T00:34:08.727+0000 7f8ac67d2640 -1 osd.1 96 *** Got signal Terminated *** 2024-09-11T00:34:09.187 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1[40006]: 2024-09-11T00:34:08.727+0000 7f8ac67d2640 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:14.363 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:14 smithi117 podman[69788]: 2024-09-11 00:34:14.0860416 +0000 UTC m=+5.502803200 container died 3b3b2fa3f6bca86bd30051d143d4086895216148b06a93e5799344d8e6d138e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:14.687 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:14 smithi117 podman[69788]: 2024-09-11 00:34:14.452141105 +0000 UTC m=+5.868902705 container cleanup 3b3b2fa3f6bca86bd30051d143d4086895216148b06a93e5799344d8e6d138e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:14.687 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:14 smithi117 bash[69788]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1 2024-09-11T00:34:15.063 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:14 smithi117 podman[69804]: 2024-09-11 00:34:14.737235321 +0000 UTC m=+0.648070170 container remove 3b3b2fa3f6bca86bd30051d143d4086895216148b06a93e5799344d8e6d138e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T00:34:15.342 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:15 smithi117 podman[69895]: 2024-09-11 00:34:14.965056375 +0000 UTC m=+0.015699087 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:15.342 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:15 smithi117 podman[69895]: 2024-09-11 00:34:15.088114992 +0000 UTC m=+0.138757691 container create ea995d59ea82c33098ff3dcfaa12d5ab620b3332077725708f0f21726a86adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:34:15.342 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:15 smithi117 podman[69895]: 2024-09-11 00:34:15.340743513 +0000 UTC m=+0.391386215 container init ea995d59ea82c33098ff3dcfaa12d5ab620b3332077725708f0f21726a86adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T00:34:15.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:15 smithi117 podman[69895]: 2024-09-11 00:34:15.344323011 +0000 UTC m=+0.394965710 container start ea995d59ea82c33098ff3dcfaa12d5ab620b3332077725708f0f21726a86adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0) 2024-09-11T00:34:15.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:15 smithi117 podman[69895]: 2024-09-11 00:34:15.449121126 +0000 UTC m=+0.499763845 container attach ea995d59ea82c33098ff3dcfaa12d5ab620b3332077725708f0f21726a86adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T00:34:15.688 INFO:journalctl@ceph.osd.1.smithi117.stdout:Sep 11 00:34:15 smithi117 podman[69895]: 2024-09-11 00:34:15.535345611 +0000 UTC m=+0.585988311 container died ea995d59ea82c33098ff3dcfaa12d5ab620b3332077725708f0f21726a86adf3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T00:34:16.166 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T00:34:16.204 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:16.205 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T00:34:16.205 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T00:34:16.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.2 2024-09-11T00:34:16.503 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:16 smithi183 systemd[1]: Stopping Ceph osd.2 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:16.767 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2[40887]: 2024-09-11T00:34:16.500+0000 7f1f07478640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:16.768 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2[40887]: 2024-09-11T00:34:16.500+0000 7f1f07478640 -1 osd.2 96 *** Got signal Terminated *** 2024-09-11T00:34:16.768 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2[40887]: 2024-09-11T00:34:16.500+0000 7f1f07478640 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:22.219 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:21 smithi183 podman[70891]: 2024-09-11 00:34:21.851306839 +0000 UTC m=+5.453375862 container died 1b6a34b7be7b13cf78402e4844b9aa8706849890bd2518f7c22d1231271b6c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:22.531 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:22 smithi183 podman[70891]: 2024-09-11 00:34:22.417908846 +0000 UTC m=+6.019977876 container cleanup 1b6a34b7be7b13cf78402e4844b9aa8706849890bd2518f7c22d1231271b6c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T00:34:22.531 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:22 smithi183 bash[70891]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2 2024-09-11T00:34:23.030 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:22 smithi183 podman[70918]: 2024-09-11 00:34:22.736000561 +0000 UTC m=+0.879351970 container remove 1b6a34b7be7b13cf78402e4844b9aa8706849890bd2518f7c22d1231271b6c0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T00:34:23.357 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:23 smithi183 podman[71002]: 2024-09-11 00:34:22.980414647 +0000 UTC m=+0.015681533 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:23.357 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:23 smithi183 podman[71002]: 2024-09-11 00:34:23.095244013 +0000 UTC m=+0.130510889 container create 39769b4e4db3a83458d844fba224fc2ab2c490be9c2a92a0aba952ec53963dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T00:34:23.357 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:23 smithi183 podman[71002]: 2024-09-11 00:34:23.356122181 +0000 UTC m=+0.391389058 container init 39769b4e4db3a83458d844fba224fc2ab2c490be9c2a92a0aba952ec53963dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:34:23.781 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:23 smithi183 podman[71002]: 2024-09-11 00:34:23.359606203 +0000 UTC m=+0.394873082 container start 39769b4e4db3a83458d844fba224fc2ab2c490be9c2a92a0aba952ec53963dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T00:34:23.781 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:23 smithi183 podman[71002]: 2024-09-11 00:34:23.456238805 +0000 UTC m=+0.491505689 container attach 39769b4e4db3a83458d844fba224fc2ab2c490be9c2a92a0aba952ec53963dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:23.781 INFO:journalctl@ceph.osd.2.smithi183.stdout:Sep 11 00:34:23 smithi183 podman[71002]: 2024-09-11 00:34:23.550195891 +0000 UTC m=+0.585462769 container died 39769b4e4db3a83458d844fba224fc2ab2c490be9c2a92a0aba952ec53963dcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:24.238 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T00:34:24.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:24.278 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T00:34:24.278 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T00:34:24.279 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.3 2024-09-11T00:34:24.670 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:24 smithi103 systemd[1]: Stopping Ceph osd.3 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:25.170 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:24 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:34:24.704+0000 7f75d0a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:25.170 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:24 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:34:24.704+0000 7f75d0a00640 -1 osd.3 96 *** Got signal Terminated *** 2024-09-11T00:34:25.170 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:24 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:34:24.704+0000 7f75d0a00640 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:27.170 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:26 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:34:26.907+0000 7f75cc400640 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:01.414631+0000 front 2024-09-11T00:34:01.414765+0000 (oldest deadline 2024-09-11T00:34:26.714652+0000) 2024-09-11T00:34:27.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:26.898+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:28.219 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:27 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:34:27.940+0000 7f75cc400640 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:01.414631+0000 front 2024-09-11T00:34:01.414765+0000 (oldest deadline 2024-09-11T00:34:26.714652+0000) 2024-09-11T00:34:28.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:27.873+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:28.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:28 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:28.217+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:28.877 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:28.592+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:29.170 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:28 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3[53271]: 2024-09-11T00:34:28.946+0000 7f75cc400640 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:01.414631+0000 front 2024-09-11T00:34:01.414765+0000 (oldest deadline 2024-09-11T00:34:26.714652+0000) 2024-09-11T00:34:29.280 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:28.876+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:29.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:29 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:29.238+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:29.914 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:29.620+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:29.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:29 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:29.510+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:30.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:29.912+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:30.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:30.042+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:30.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:30 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:30.257+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:30.671 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:30 smithi103 podman[116644]: 2024-09-11 00:34:30.328479744 +0000 UTC m=+5.754125502 container died 012235ba4d2eca7f5bda33c8e795c6c34bed4a00c4fd0315efdac8918ac75351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T00:34:30.927 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:30.630+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:30.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:30 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:30.485+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:31.163 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:30 smithi103 podman[116644]: 2024-09-11 00:34:30.818205156 +0000 UTC m=+6.243850914 container cleanup 012235ba4d2eca7f5bda33c8e795c6c34bed4a00c4fd0315efdac8918ac75351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:31.163 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:30 smithi103 bash[116644]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3 2024-09-11T00:34:31.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:31.065+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:31.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:30.925+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:31.419 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:31 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:31.241+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:31.420 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:31 smithi103 podman[116660]: 2024-09-11 00:34:31.161331349 +0000 UTC m=+0.822431042 container remove 012235ba4d2eca7f5bda33c8e795c6c34bed4a00c4fd0315efdac8918ac75351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T00:34:31.920 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:31 smithi103 podman[116749]: 2024-09-11 00:34:31.502077242 +0000 UTC m=+0.027554363 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:31.920 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:31 smithi103 podman[116749]: 2024-09-11 00:34:31.671330303 +0000 UTC m=+0.196807422 container create 400718e7b5ace916c09b9ed2a548c2d4a4488eb9a6e2aa9ef5f22cde0e0371a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:34:31.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:31 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:31.446+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:31.973 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:31.665+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:32.218 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:31 smithi103 podman[116749]: 2024-09-11 00:34:31.948893339 +0000 UTC m=+0.474370460 container init 400718e7b5ace916c09b9ed2a548c2d4a4488eb9a6e2aa9ef5f22cde0e0371a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T00:34:32.218 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:31 smithi103 podman[116749]: 2024-09-11 00:34:31.955680094 +0000 UTC m=+0.481157224 container start 400718e7b5ace916c09b9ed2a548c2d4a4488eb9a6e2aa9ef5f22cde0e0371a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T00:34:32.218 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:32 smithi103 podman[116749]: 2024-09-11 00:34:32.115568101 +0000 UTC m=+0.641045237 container attach 400718e7b5ace916c09b9ed2a548c2d4a4488eb9a6e2aa9ef5f22cde0e0371a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T00:34:32.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:32.092+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:32.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:31.970+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:32.594 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:32 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:32.284+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:32.595 INFO:journalctl@ceph.osd.3.smithi103.stdout:Sep 11 00:34:32 smithi103 podman[116749]: 2024-09-11 00:34:32.217656407 +0000 UTC m=+0.743133546 container died 400718e7b5ace916c09b9ed2a548c2d4a4488eb9a6e2aa9ef5f22cde0e0371a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-11T00:34:32.687 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:32 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:32.410+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:32.982 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:32.631+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:33.009 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T00:34:33.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:33.085 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T00:34:33.085 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T00:34:33.085 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.4 2024-09-11T00:34:33.140 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:33 smithi117 systemd[1]: Stopping Ceph osd.4 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:33.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:32.980+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:33.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:33.105+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:33.669 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:33 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:33.307+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:33.687 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:33 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:33.441+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:33.687 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:33 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:34:33.397+0000 7fb129335640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:33.687 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:33 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:34:33.397+0000 7fb129335640 -1 osd.4 96 *** Got signal Terminated *** 2024-09-11T00:34:33.687 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:33 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:34:33.397+0000 7fb129335640 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:34.021 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:33.641+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:34.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:34.019+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:34.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:34.154+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:34.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:34 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:34.271+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:34.913 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:34 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:34.470+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:35.012 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:34.607+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:35.187 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:34 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:34:34.911+0000 7fb121942640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:11.286114+0000 front 2024-09-11T00:34:11.286068+0000 (oldest deadline 2024-09-11T00:34:34.786138+0000) 2024-09-11T00:34:35.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:35.139+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:35.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:35.009+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:35.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:35.009+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:35.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:35 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:35.263+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:35.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:35 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:35.263+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:35.878 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:35 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:35.452+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:36.030 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:35.614+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:36.187 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:35 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:34:35.877+0000 7fb121942640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:11.286114+0000 front 2024-09-11T00:34:11.286068+0000 (oldest deadline 2024-09-11T00:34:34.786138+0000) 2024-09-11T00:34:36.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:36 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:36.187+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:36.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:36 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:36.187+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:36.531 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:36 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:36.029+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:36.531 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:36 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:36.029+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:36.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:36 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:36.241+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:36.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:36 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:36.241+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:36.863 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:36 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:36.502+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:37.030 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:36 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:36.634+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:37.187 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:36 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:34:36.862+0000 7fb121942640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:11.286114+0000 front 2024-09-11T00:34:11.286068+0000 (oldest deadline 2024-09-11T00:34:34.786138+0000) 2024-09-11T00:34:37.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:37.028+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:37.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:37.028+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:37.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:37.165+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:37.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:37.165+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:37.669 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:37 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:37.229+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:37.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:37 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:37.229+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:37.894 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:37 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:37.466+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:37.894 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:37 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:37.608+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:37.981 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:37.618+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:38.187 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:37 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4[46453]: 2024-09-11T00:34:37.892+0000 7fb121942640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:11.286114+0000 front 2024-09-11T00:34:11.286068+0000 (oldest deadline 2024-09-11T00:34:34.786138+0000) 2024-09-11T00:34:38.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:37.979+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:38.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:37 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:37.979+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:38.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:38.205+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:38.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:38.205+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:38.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:38 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:38.247+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:38.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:38 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:38.247+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:38.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:38 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:38.397+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:38.915 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:38.575+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:38.915 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:38.476+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:38.915 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:38 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:38.476+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:38.978 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:38.653+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:39.187 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:38 smithi117 podman[70010]: 2024-09-11 00:34:38.913630916 +0000 UTC m=+5.630283956 container died 9e2ba897770f85394850f2160ff20636b7074f5570ebd3aaa305db5c24d2c622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:39.255 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:38.976+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:39.255 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:38 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:38.976+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:39.518 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:39 smithi117 podman[70010]: 2024-09-11 00:34:39.288188857 +0000 UTC m=+6.004841896 container cleanup 9e2ba897770f85394850f2160ff20636b7074f5570ebd3aaa305db5c24d2c622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:39.518 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:39 smithi117 bash[70010]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4 2024-09-11T00:34:39.530 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:39 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:39.253+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:39.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:39 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:39.253+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:39.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:39 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:39.363+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:39.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:39 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:39.266+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:39.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:39 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:39.266+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:39.907 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:39 smithi117 podman[70027]: 2024-09-11 00:34:39.573154113 +0000 UTC m=+0.655046533 container remove 9e2ba897770f85394850f2160ff20636b7074f5570ebd3aaa305db5c24d2c622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-11T00:34:39.907 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:39 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:39.567+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:39.907 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:39 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:39.514+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:39.907 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:39 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:39.514+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:40.021 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:39 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:39.631+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:40.186 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:39 smithi117 podman[70117]: 2024-09-11 00:34:39.80837385 +0000 UTC m=+0.015551700 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:40.186 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:39 smithi117 podman[70117]: 2024-09-11 00:34:39.940959958 +0000 UTC m=+0.148137796 container create 27dfaa180ff2db26fbc0eb65d77241c6feebe6ce01d91b1511c8642bd6f3c4dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T00:34:40.186 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:40 smithi117 podman[70117]: 2024-09-11 00:34:40.184921027 +0000 UTC m=+0.392098864 container init 27dfaa180ff2db26fbc0eb65d77241c6feebe6ce01d91b1511c8642bd6f3c4dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T00:34:40.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:40 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:40.019+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:40.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:40 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:40.019+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:40.437 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:40 smithi117 podman[70117]: 2024-09-11 00:34:40.18827923 +0000 UTC m=+0.395457068 container start 27dfaa180ff2db26fbc0eb65d77241c6feebe6ce01d91b1511c8642bd6f3c4dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-11T00:34:40.438 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:40 smithi117 podman[70117]: 2024-09-11 00:34:40.251717481 +0000 UTC m=+0.458895330 container attach 27dfaa180ff2db26fbc0eb65d77241c6feebe6ce01d91b1511c8642bd6f3c4dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T00:34:40.438 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 11 00:34:40 smithi117 podman[70117]: 2024-09-11 00:34:40.374599109 +0000 UTC m=+0.581776961 container died 27dfaa180ff2db26fbc0eb65d77241c6feebe6ce01d91b1511c8642bd6f3c4dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-11T00:34:40.584 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:40 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:40.297+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:40.584 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:40 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:40.297+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:40.669 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:40 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:40.368+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:40.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:40 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:40.218+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:40.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:40 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:40.218+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:40.923 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:40 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:40.477+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:40.923 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:40 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:40.477+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:40.923 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:40 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:40.607+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:40.940 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T00:34:40.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:40.979 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T00:34:40.979 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T00:34:40.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.5 2024-09-11T00:34:41.006 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:40 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:40.582+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:41.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:41.015+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:41.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:41.015+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:41.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 systemd[1]: Stopping Ceph osd.5 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:41.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:41.310+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:41.548 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:41.310+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:41.548 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:41.278+0000 7f01b39f4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:41.548 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:41.278+0000 7f01b39f4640 -1 osd.5 96 *** Got signal Terminated *** 2024-09-11T00:34:41.548 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:41.278+0000 7f01b39f4640 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:41.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:41 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:41.359+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:41.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:41 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:41.179+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:41.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:41 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:41.179+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:41.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:41 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:41.634+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:41.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:41 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:41.500+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:41.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:41 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:41.500+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:41.988 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:41.546+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:42.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:41.986+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:42.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:41 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:41.986+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:42.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:42 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:42.359+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:42.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:42 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:42.179+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:42.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:42 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:42.179+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:42.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:42.540+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:42.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:42.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:42.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:42.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:42.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:42.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:42.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:42 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:42.588+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:42.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:42 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:42.550+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:42.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:42 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:42.550+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:43.272 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:42.990+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:43.272 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:42 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:42.990+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:43.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:43 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:43.348+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:43.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:43 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:43.167+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:43.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:43 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:43.167+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:43.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:43 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:43.167+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:43.525 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:43 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:43.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:43.526 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:43 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:43.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:43.526 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:43 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:43.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:43.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:43 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:43.523+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:43.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:43 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:43.615+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:43.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:43 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:43.539+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:43.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:43 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:43.539+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:44.271 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:43 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:43.971+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:44.271 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:43 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:43.971+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:44.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:44 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:44.335+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:44.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:44 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:44.335+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:44.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:44 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:44.165+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:44.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:44 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:44.165+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:44.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:44 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:44.165+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:44.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:44 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:44.486+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:44.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:44 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:44.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:44.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:44 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:44.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:44.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:44 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:44.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:44.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:44 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:44.615+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:44.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:44 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:44.615+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:44.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:44 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:44.529+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:44.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:44 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:44.529+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:44.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:44 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:44.529+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:45.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:44 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:44.947+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:45.281 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:44 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:44.947+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:45.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:45 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:45.379+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:45.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:45 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:45.379+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:45.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:45 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:45.157+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:45.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:45 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:45.157+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:45.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:45 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:45.157+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:45.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:45 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:45.442+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:45.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:45 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:45.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:45.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:45 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:45.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:45.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:45 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:45.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:45.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:45 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:45.580+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:45.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:45 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:45.580+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:45.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:45 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:45.496+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:45.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:45 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:45.496+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:45.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:45 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:45.496+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:46.263 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:45 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:45.977+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.437261+0000 front 2024-09-11T00:34:04.437436+0000 (oldest deadline 2024-09-11T00:34:26.137305+0000) 2024-09-11T00:34:46.263 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:45 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:45.977+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.039350+0000 front 2024-09-11T00:34:13.039465+0000 (oldest deadline 2024-09-11T00:34:34.739319+0000) 2024-09-11T00:34:46.263 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:45 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5[47502]: 2024-09-11T00:34:45.977+0000 7f01b0009640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.841378+0000 front 2024-09-11T00:34:19.841341+0000 (oldest deadline 2024-09-11T00:34:45.141540+0000) 2024-09-11T00:34:46.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:46 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:46.449+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:46.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:46 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:46.449+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:46.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:46 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:46.261+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:46.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:46 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:46.261+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:46.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:46 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:46.261+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:46.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:46 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:46.412+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:46.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:46 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:46.412+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:46.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:46 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:46.196+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:46.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:46 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:46.196+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:46.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:46 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:46.196+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:46.839 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:46 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:46.623+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:46.839 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:46 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:46.623+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:46.839 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:46 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:46.516+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:46.839 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:46 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:46.516+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:46.839 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:46 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:46.516+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:47.189 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:46 smithi183 podman[71103]: 2024-09-11 00:34:46.884845929 +0000 UTC m=+5.718060356 container died 19bccb2d639ab3549094bf01de2da8e465d4119379208df37fe75bafb976e667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:47.454 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:47 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:47.192+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:47.454 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:47 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:47.192+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:47.454 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:47 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:47.192+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:47.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:47 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:47.430+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:47.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:47 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:47.430+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:47.531 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:47 smithi183 podman[71103]: 2024-09-11 00:34:47.295807041 +0000 UTC m=+6.129021465 container cleanup 19bccb2d639ab3549094bf01de2da8e465d4119379208df37fe75bafb976e667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T00:34:47.531 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:47 smithi183 bash[71103]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5 2024-09-11T00:34:47.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:47 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:47.302+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:47.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:47 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:47.302+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:47.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:47 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:47.302+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:47.919 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:47 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:47.452+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:47.920 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:47 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:47.452+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:47.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:47 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:47.601+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:47.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:47 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:47.601+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:47.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:47 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:47.536+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:47.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:47 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:47.536+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:47.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:47 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:47.536+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:47.970 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:47 smithi183 podman[71118]: 2024-09-11 00:34:47.555441021 +0000 UTC m=+0.663533770 container remove 19bccb2d639ab3549094bf01de2da8e465d4119379208df37fe75bafb976e667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T00:34:48.227 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:47 smithi183 podman[71208]: 2024-09-11 00:34:47.872024516 +0000 UTC m=+0.015501066 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:48.227 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:47 smithi183 podman[71208]: 2024-09-11 00:34:47.981210751 +0000 UTC m=+0.124687289 container create 6b2c85fe09e3e00986de54ce77a3919c2d88b623606705c2e87f98d99ca70a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:48.227 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:48 smithi183 podman[71208]: 2024-09-11 00:34:48.225882499 +0000 UTC m=+0.369359048 container init 6b2c85fe09e3e00986de54ce77a3919c2d88b623606705c2e87f98d99ca70a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T00:34:48.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:48 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:48.430+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:48.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:48 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:48.430+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:48.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:48 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:48.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:48.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:48 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:48.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:48.532 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:48 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:48.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:48.532 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:48 smithi183 podman[71208]: 2024-09-11 00:34:48.229782556 +0000 UTC m=+0.373259094 container start 6b2c85fe09e3e00986de54ce77a3919c2d88b623606705c2e87f98d99ca70a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:48.532 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:48 smithi183 podman[71208]: 2024-09-11 00:34:48.33370895 +0000 UTC m=+0.477185496 container attach 6b2c85fe09e3e00986de54ce77a3919c2d88b623606705c2e87f98d99ca70a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-11T00:34:48.532 INFO:journalctl@ceph.osd.5.smithi183.stdout:Sep 11 00:34:48 smithi183 podman[71208]: 2024-09-11 00:34:48.415170562 +0000 UTC m=+0.558647109 container died 6b2c85fe09e3e00986de54ce77a3919c2d88b623606705c2e87f98d99ca70a5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:48.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:48 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:48.460+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:48.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:48 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:48.460+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:48.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:48 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:48.220+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:48.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:48 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:48.220+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:48.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:48 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:48.220+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:48.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:48 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:48.618+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:48.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:48 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:48.618+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:48.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:48 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:48.528+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:48.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:48 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:48.528+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:48.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:48 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:48.528+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:49.057 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T00:34:49.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:49.138 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T00:34:49.138 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T00:34:49.139 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.6 2024-09-11T00:34:49.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:49 smithi103 systemd[1]: Stopping Ceph osd.6 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:49.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:49.217+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:49.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:49.217+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:49.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:49.217+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:49.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:49 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:49.401+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:49.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:49 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:49.401+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:49.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:49 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:49.366+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:49.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:49 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:49.366+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:49.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:49 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:49.366+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:49.809 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:49.458+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:49.809 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:49.458+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:49.809 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:49.567+0000 7fb237600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:49.809 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:49.567+0000 7fb237600640 -1 osd.6 96 *** Got signal Terminated *** 2024-09-11T00:34:49.809 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:49 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:49.567+0000 7fb237600640 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:49.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:49 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:49.587+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:49.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:49 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:49.587+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:49.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:49 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:49.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:49.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:49 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:49.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:49.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:49 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:49.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:50.640 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:50 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:50.422+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:50.640 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:50 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:50.422+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:50.640 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:50 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:50.251+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:50.640 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:50 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:50.251+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:50.640 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:50 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:50.251+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:50.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:50 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:50.434+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:50.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:50 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:50.434+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:50.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:50 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:50.394+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:50.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:50 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:50.394+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:50.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:50 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:50.394+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:50.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:50 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:50.603+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:50.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:50 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:50.603+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:50.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:50 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:50.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:50.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:50 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:50.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:50.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:50 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:50.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:51.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:51.453+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:51.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:51.453+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:51.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:51.453+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.540192+0000 front 2024-09-11T00:34:28.540388+0000 (oldest deadline 2024-09-11T00:34:50.840207+0000) 2024-09-11T00:34:51.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:51.256+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:51.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:51.256+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:51.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:51 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:51.256+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:51.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:51 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:51.469+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:51.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:51 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:51.469+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:51.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:51 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:51.469+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:51.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:51 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:51.379+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:51.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:51 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:51.379+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:51.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:51 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:51.379+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:51.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:51 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:51.525+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:51.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:51 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:51.525+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:51.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:51 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:51.525+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:51.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:51 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:51.611+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:51.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:51 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:51.611+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:51.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:51 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:51.611+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:52.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:52.447+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:52.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:52.447+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:52.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:52.447+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.540192+0000 front 2024-09-11T00:34:28.540388+0000 (oldest deadline 2024-09-11T00:34:50.840207+0000) 2024-09-11T00:34:52.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:52.248+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:52.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:52.248+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:52.671 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:52.248+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:52.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:52 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:52.472+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:52.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:52 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:52.472+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:52.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:52 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:52.472+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:52.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:52 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:52.380+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:52.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:52 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:52.380+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:52.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:52 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:52.380+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:52.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:52.498+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:52.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:52.498+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:52.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:52.498+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:52.939 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:52.570+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:52.939 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:52.570+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:52.939 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:52 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:52.570+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:53.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:53 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:53.423+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:53.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:53 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:53.423+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:53.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:53 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:53.423+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.540192+0000 front 2024-09-11T00:34:28.540388+0000 (oldest deadline 2024-09-11T00:34:50.840207+0000) 2024-09-11T00:34:53.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:53 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:53.250+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:53.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:53 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:53.250+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:53.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:53 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:53.250+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:53.671 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:53 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:53.250+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:34:53.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:53.476+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:53.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:53.476+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:53.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:53.476+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:53.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:53.367+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:53.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:53.367+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:53.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:53 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:53.367+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:53.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:53 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:53.544+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:53.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:53 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:53.544+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:53.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:53 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:53.544+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:53.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:53 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:53.609+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:53.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:53 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:53.609+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:53.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:53 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:53.609+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:54.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:54 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:54.442+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.936341+0000 front 2024-09-11T00:34:12.936116+0000 (oldest deadline 2024-09-11T00:34:37.636300+0000) 2024-09-11T00:34:54.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:54 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:54.442+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.737484+0000 front 2024-09-11T00:34:18.737597+0000 (oldest deadline 2024-09-11T00:34:44.037692+0000) 2024-09-11T00:34:54.670 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:54 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6[59741]: 2024-09-11T00:34:54.442+0000 7fb233000640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.540192+0000 front 2024-09-11T00:34:28.540388+0000 (oldest deadline 2024-09-11T00:34:50.840207+0000) 2024-09-11T00:34:54.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:54 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:54.237+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:54.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:54 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:54.237+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:54.671 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:54 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:54.237+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:54.671 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:54 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:54.237+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:34:54.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:54 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:54.526+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:54.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:54 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:54.526+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:54.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:54 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:54.526+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:54.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:54 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:54.406+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:54.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:54 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:54.406+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:54.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:54 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:54.406+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:54.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:54 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:54.593+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:54.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:54 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:54.593+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:54.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:54 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:54.593+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:54.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:54 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:54.497+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:54.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:54 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:54.497+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:54.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:54 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:54.497+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:54.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:54 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:54.497+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:34:55.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:55 smithi103 podman[116851]: 2024-09-11 00:34:55.080491504 +0000 UTC m=+5.646542990 container died 873e73886f5191ae8a09c5baadf81deaccbcadeefc9df1e49260d25a5721b292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:34:55.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:55 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:55.191+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:55.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:55 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:55.191+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:55.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:55 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:55.191+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:55.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:55 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:55.191+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:34:55.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:55 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:55.569+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:55.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:55 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:55.569+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:55.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:55 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:55.569+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:55.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:55 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:55.360+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:55.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:55 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:55.360+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:55.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:55 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:55.360+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:55.862 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:55 smithi103 podman[116851]: 2024-09-11 00:34:55.597994909 +0000 UTC m=+6.164046403 container cleanup 873e73886f5191ae8a09c5baadf81deaccbcadeefc9df1e49260d25a5721b292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-11T00:34:55.862 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:55 smithi103 bash[116851]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6 2024-09-11T00:34:55.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:55.615+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:55.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:55.615+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:55.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:55.615+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:55.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:55.468+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:55.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:55.468+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:55.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:55.468+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:55.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:55 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:55.468+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:34:56.170 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:55 smithi103 podman[116878]: 2024-09-11 00:34:55.860825683 +0000 UTC m=+0.761857813 container remove 873e73886f5191ae8a09c5baadf81deaccbcadeefc9df1e49260d25a5721b292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T00:34:56.468 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:56 smithi103 podman[116970]: 2024-09-11 00:34:56.32765109 +0000 UTC m=+0.027443968 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:34:56.468 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:56 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:56.210+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:56.469 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:56 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:56.210+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:56.469 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:56 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:56.210+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:56.469 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:56 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:56.210+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:34:56.573 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:56 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:56.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:56.573 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:56 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:56.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:56.573 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:56 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:56.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:56.771 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:56 smithi103 podman[116970]: 2024-09-11 00:34:56.467273408 +0000 UTC m=+0.167066276 container create 53e1e7c8bb886e4b19d6c308e74a19b3c36a96c800a59234da84ab23ab5d7225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T00:34:56.771 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:56 smithi103 podman[116970]: 2024-09-11 00:34:56.770065405 +0000 UTC m=+0.469858287 container init 53e1e7c8bb886e4b19d6c308e74a19b3c36a96c800a59234da84ab23ab5d7225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:34:56.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:56 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:56.567+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:56.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:56 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:56.567+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:56.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:56 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:56.567+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:56.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:56 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:56.496+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:56.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:56 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:56.496+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:56.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:56 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:56.496+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:56.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:56 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:56.496+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:34:57.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:56 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:56.571+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:57.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:56 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:56.571+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:57.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:56 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:56.571+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:57.041 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:56 smithi103 podman[116970]: 2024-09-11 00:34:56.77719523 +0000 UTC m=+0.476988101 container start 53e1e7c8bb886e4b19d6c308e74a19b3c36a96c800a59234da84ab23ab5d7225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:34:57.041 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:56 smithi103 podman[116970]: 2024-09-11 00:34:56.898081056 +0000 UTC m=+0.597873940 container attach 53e1e7c8bb886e4b19d6c308e74a19b3c36a96c800a59234da84ab23ab5d7225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:34:57.420 INFO:journalctl@ceph.osd.6.smithi103.stdout:Sep 11 00:34:57 smithi103 podman[116970]: 2024-09-11 00:34:57.040377736 +0000 UTC m=+0.740170606 container died 53e1e7c8bb886e4b19d6c308e74a19b3c36a96c800a59234da84ab23ab5d7225 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T00:34:57.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:57 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:57.242+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:57.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:57 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:57.242+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:57.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:57 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:57.242+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:57.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:57 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:57.242+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:34:57.584 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:57 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:57.320+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:57.584 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:57 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:57.320+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:57.584 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:57 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:57.320+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:57.860 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T00:34:57.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:34:57.935 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T00:34:57.936 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T00:34:57.936 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.7 2024-09-11T00:34:57.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:57.552+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:57.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:57.552+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:57.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:57.552+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:57.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:57.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:57.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:57.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:57.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:57.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:57.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:57.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:34:58.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:57 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:57.582+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:58.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:57 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:57.582+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:58.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:57 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:57.582+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:58.258 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:57 smithi117 systemd[1]: Stopping Ceph osd.7 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:34:58.258 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:58.256+0000 7ff387a1c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:34:58.259 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:58.256+0000 7ff387a1c640 -1 osd.7 96 *** Got signal Terminated *** 2024-09-11T00:34:58.259 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:58.256+0000 7ff387a1c640 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:34:58.541 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:58 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:58.283+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:58.541 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:58 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:58.283+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:58.541 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:58 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:58.283+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:58.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:58 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:58.248+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:58.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:58 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:58.248+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:58.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:58 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:58.248+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:58.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:58 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:58.248+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:34:58.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:58.602+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:58.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:58.602+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:58.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:58.602+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:58.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:58.534+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:58.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:58.534+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:58.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:58.534+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:58.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:58 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:58.534+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:34:59.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:58 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:58.539+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:59.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:58 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:58.539+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:59.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:58 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:58.539+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:59.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:59.267+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:34:59.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:59.267+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:34:59.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:59.267+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:34:59.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:34:59 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:34:59.267+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:34:59.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:59 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:59.564+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:34:59.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:59 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:59.564+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:34:59.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:34:59 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:34:59.564+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:34:59.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:59 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:59.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:34:59.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:59 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:59.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:34:59.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:34:59 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:34:59.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:34:59.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:59 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:59.639+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:34:59.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:59 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:59.639+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:34:59.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:34:59 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:34:59.639+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:34:59.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:59 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:59.510+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:34:59.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:59 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:59.510+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:34:59.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:59 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:59.510+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:34:59.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:34:59 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:34:59.510+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:00.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:00 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:00.276+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:00.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:00 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:00.276+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:00.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:00 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:00.276+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:00.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:00 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:00.276+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:00.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:00 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:00.363+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:00.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:00 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:00.363+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:00.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:00 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:00.363+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:00.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:00 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:00.574+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:00.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:00 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:00.574+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:00.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:00 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:00.574+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:00.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:00 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:00.574+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:00.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:00.654+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:00.654+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:00.654+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:00.654+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.318047+0000 front 2024-09-11T00:34:38.318065+0000 (oldest deadline 2024-09-11T00:35:00.618131+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:00.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:00.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:00.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:00.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:00.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:00 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:00.547+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:01.623 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:01 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:01.354+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:01.623 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:01 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:01.354+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:01.623 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:01 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:01.354+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:01.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:01 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:01.272+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:01.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:01 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:01.272+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:01.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:01 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:01.272+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:01.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:01 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:01.272+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:01.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:01.612+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:35:01.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:01.612+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:35:01.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:01.612+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:35:01.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:01.612+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.318047+0000 front 2024-09-11T00:34:38.318065+0000 (oldest deadline 2024-09-11T00:35:00.618131+0000) 2024-09-11T00:35:01.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:01.511+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:01.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:01.511+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:01.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:01.511+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:01.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:01.511+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:01.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:01 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:01.511+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:02.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:01 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:01.621+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:02.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:01 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:01.621+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:02.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:01 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:01.621+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:02.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:01 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:01.621+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:02.609 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:02.357+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:02.610 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:02.357+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:02.610 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:02.357+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:02.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:02 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:02.269+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:02.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:02 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:02.269+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:02.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:02 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:02.269+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:02.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:02 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:02.269+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:02.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:02 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:02.269+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:02.937 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:02.639+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.114064+0000 front 2024-09-11T00:34:12.114094+0000 (oldest deadline 2024-09-11T00:34:37.414276+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:02.639+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:19.615488+0000 front 2024-09-11T00:34:19.615676+0000 (oldest deadline 2024-09-11T00:34:44.315510+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:02.639+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:25.416510+0000 front 2024-09-11T00:34:25.416415+0000 (oldest deadline 2024-09-11T00:34:51.316455+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7[53404]: 2024-09-11T00:35:02.639+0000 7ff380029640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.318047+0000 front 2024-09-11T00:34:38.318065+0000 (oldest deadline 2024-09-11T00:35:00.618131+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:02.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:02.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:02.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:02.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:02.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:02.541+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:03.030 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:02.608+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:03.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:02.608+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:03.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:02.608+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:03.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:02 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:02.608+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:03.654 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:03 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:03.342+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:03.654 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:03 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:03.342+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:03.654 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:03 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:03.342+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:03.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:03.304+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:03.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:03.304+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:03.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:03.304+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:03.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:03.304+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:03.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:03 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:03.304+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:03.832 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:03 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:03.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:03.833 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:03 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:03.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:03.833 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:03 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:03.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:03.833 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:03 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:03.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:03.833 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:03 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:03.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:04.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:03 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:03.652+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:04.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:03 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:03.652+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:04.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:03 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:03.652+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:04.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:03 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:03.652+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:04.126 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:03 smithi117 podman[70234]: 2024-09-11 00:35:03.830922414 +0000 UTC m=+5.693963347 container died 4dde8bfa0d7713d266885c7e88f54ee586ee747eaee9899a667daf2ab4f0acb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T00:35:04.437 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:04 smithi117 podman[70234]: 2024-09-11 00:35:04.206326224 +0000 UTC m=+6.069367156 container cleanup 4dde8bfa0d7713d266885c7e88f54ee586ee747eaee9899a667daf2ab4f0acb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T00:35:04.438 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:04 smithi117 bash[70234]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7 2024-09-11T00:35:04.662 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:04.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:04.663 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:04.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:04.663 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:04.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:04.663 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:04.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:04.669 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:04 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:04.270+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:04.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:04 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:04.270+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:04.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:04 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:04.270+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:04.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:04 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:04.270+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:04.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:04 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:04.270+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:04.784 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:04 smithi117 podman[70248]: 2024-09-11 00:35:04.482593262 +0000 UTC m=+0.646749798 container remove 4dde8bfa0d7713d266885c7e88f54ee586ee747eaee9899a667daf2ab4f0acb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T00:35:04.784 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:04 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:04.537+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:04.784 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:04 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:04.537+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:04.784 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:04 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:04.537+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:04.785 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:04 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:04.537+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:04.785 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:04 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:04.537+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:05.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:04.661+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:05.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:04.661+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:05.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:04.661+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:05.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:04 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:04.661+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:05.036 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:04 smithi117 podman[70337]: 2024-09-11 00:35:04.783181589 +0000 UTC m=+0.108820241 container create a3ea5f2f3bc0696eb934999ed88372314b34855d7817ce83a9a540382acf832b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:05.037 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:04 smithi117 podman[70337]: 2024-09-11 00:35:04.690276686 +0000 UTC m=+0.015915339 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:35:05.037 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:05 smithi117 podman[70337]: 2024-09-11 00:35:05.035851652 +0000 UTC m=+0.361490306 container init a3ea5f2f3bc0696eb934999ed88372314b34855d7817ce83a9a540382acf832b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T00:35:05.437 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:05 smithi117 podman[70337]: 2024-09-11 00:35:05.039162089 +0000 UTC m=+0.364800746 container start a3ea5f2f3bc0696eb934999ed88372314b34855d7817ce83a9a540382acf832b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-11T00:35:05.437 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:05 smithi117 podman[70337]: 2024-09-11 00:35:05.144373357 +0000 UTC m=+0.470012028 container attach a3ea5f2f3bc0696eb934999ed88372314b34855d7817ce83a9a540382acf832b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T00:35:05.437 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 11 00:35:05 smithi117 podman[70337]: 2024-09-11 00:35:05.220101483 +0000 UTC m=+0.545740137 container died a3ea5f2f3bc0696eb934999ed88372314b34855d7817ce83a9a540382acf832b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T00:35:05.637 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:05.286+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:05.637 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:05.286+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:05.637 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:05.286+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:05.637 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:05.286+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:05.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:05 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:05.238+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:05.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:05 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:05.239+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:05.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:05 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:05.239+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:05.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:05 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:05.239+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:05.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:05 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:05.239+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:05.849 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:05 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:05.492+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:05.849 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:05 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:05.492+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:05.849 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:05 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:05.492+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:05.849 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:05 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:05.492+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:05.849 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:05 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:05.492+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:05.867 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T00:35:05.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:05.906 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T00:35:05.906 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T00:35:05.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.8 2024-09-11T00:35:05.933 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:05.635+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:05.933 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:05.635+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:05.933 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:05.635+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:05.933 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:05 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:05.635+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:06.199 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:05 smithi183 systemd[1]: Stopping Ceph osd.8 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:06.530 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:06.197+0000 7fad2267b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:06.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:06.197+0000 7fad2267b640 -1 osd.8 96 *** Got signal Terminated *** 2024-09-11T00:35:06.531 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:06.197+0000 7fad2267b640 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:35:06.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:06.278+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:06.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:06.278+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:06.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:06.278+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:06.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:06.278+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:06.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:06 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:06.271+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:06.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:06 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:06.271+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:06.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:06 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:06.271+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:06.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:06 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:06.271+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:06.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:06 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:06.271+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:06.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:06 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:06.519+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:06.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:06 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:06.519+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:06.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:06 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:06.519+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:06.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:06 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:06.519+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:06.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:06 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:06.519+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:07.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:06.653+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:07.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:06.653+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:07.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:06.653+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:07.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:06 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:06.653+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:07.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:07 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:07.311+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:07.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:07 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:07.311+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:07.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:07 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:07.311+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:07.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:07 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:07.311+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:07.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:07 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:07.311+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:07.619 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:07.285+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:07.619 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:07.285+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:07.619 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:07.285+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:07.619 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:07.285+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:07.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:07 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:07.568+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:07.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:07 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:07.568+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:07.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:07 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:07.568+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:07.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:07 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:07.568+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:07.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:07 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:07.568+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:08.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:07.616+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:08.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:07.616+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:08.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:07.616+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:08.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:07 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:07.616+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:08.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:08.248+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:08.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:08.248+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:08.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:08.248+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:08.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:08.248+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:08.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:08.248+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:08.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:08 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:08.348+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:08.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:08 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:08.348+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:08.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:08 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:08.348+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:08.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:08 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:08.348+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:08.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:08 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:08.348+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:08.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:08.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:08.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:08.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:08.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:08.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:08.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:08.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:08.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:08.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:08.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:08 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:08.540+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:09.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:08.590+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:09.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:08.590+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:09.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:08.590+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:09.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:08 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:08.590+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:09.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:09.252+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:09.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:09.252+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:09.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:09.252+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:09.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:09.252+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:09.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:09.252+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:09.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:09 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:09.369+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:09.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:09 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:09.369+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:09.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:09 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:09.369+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:09.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:09 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:09.369+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:09.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:09 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:09.369+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:09.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:09 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:09.369+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:09.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:09 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:09.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:09.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:09 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:09.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:09.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:09 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:09.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:09.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:09 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:09.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:09.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:09 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:09.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:09.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:09 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:09.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:10.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:09.560+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:10.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:09.560+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:10.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:09.560+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:10.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:09 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:09.560+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:10.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:10.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:10.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:10.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:10.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:10.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:10.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:10.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:10.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:10.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:10.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:10 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:10.320+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:10.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:10 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:10.320+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:10.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:10 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:10.320+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:10.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:10 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:10.320+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:10.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:10 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:10.320+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:10.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:10 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:10.320+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:10.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:10 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:10.551+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:10.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:10 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:10.551+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:10.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:10 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:10.551+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:10.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:10 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:10.551+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:10.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:10 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:10.551+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:10.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:10 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:10.551+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:11.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:10.558+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.784794+0000 front 2024-09-11T00:34:03.784724+0000 (oldest deadline 2024-09-11T00:34:27.884834+0000) 2024-09-11T00:35:11.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:10.558+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.186858+0000 front 2024-09-11T00:34:20.186954+0000 (oldest deadline 2024-09-11T00:34:46.086968+0000) 2024-09-11T00:35:11.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:10.558+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:26.087467+0000 front 2024-09-11T00:34:26.087457+0000 (oldest deadline 2024-09-11T00:34:51.287917+0000) 2024-09-11T00:35:11.031 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:10 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8[54001]: 2024-09-11T00:35:10.558+0000 7fad1ac88640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:38.189852+0000 front 2024-09-11T00:34:38.190021+0000 (oldest deadline 2024-09-11T00:35:00.489846+0000) 2024-09-11T00:35:11.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:11 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:11.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:11.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:11 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:11.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:11.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:11 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:11.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:11.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:11 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:11.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:11.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:11 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:11.269+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:11.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:11 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:11.293+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:11.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:11 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:11.293+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:11.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:11 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:11.293+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:11.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:11 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:11.293+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:11.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:11 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:11.293+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:11.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:11 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:11.293+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:11.890 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:11 smithi183 podman[71310]: 2024-09-11 00:35:11.599688929 +0000 UTC m=+5.502975208 container died 3716f715ca77f43d7739c1abd02eb31ec350508b32f68def5a327150489e3b45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:35:11.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:11.560+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:11.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:11.560+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:11.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:11.560+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:11.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:11.560+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:11.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:11.560+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:11.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:11.560+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:12.199 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:11 smithi183 podman[71310]: 2024-09-11 00:35:11.988342904 +0000 UTC m=+5.891629182 container cleanup 3716f715ca77f43d7739c1abd02eb31ec350508b32f68def5a327150489e3b45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:12.199 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:11 smithi183 bash[71310]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8 2024-09-11T00:35:12.525 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:12 smithi183 podman[71324]: 2024-09-11 00:35:12.197786372 +0000 UTC m=+0.598818162 container remove 3716f715ca77f43d7739c1abd02eb31ec350508b32f68def5a327150489e3b45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T00:35:12.525 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:12 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:12.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:12.526 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:12 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:12.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:12.526 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:12 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:12.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:12.526 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:12 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:12.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:12.526 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:12 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:12.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:12.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:12 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:12.314+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:12.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:12 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:12.314+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:12.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:12 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:12.314+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:12.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:12 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:12.314+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:12.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:12 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:12.314+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:12.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:12 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:12.314+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:12.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:12 smithi183 podman[71412]: 2024-09-11 00:35:12.523618433 +0000 UTC m=+0.112794599 container create 6e635d8921263505db1340361e7a1b5b07c89467029dd9c55825d86e47f6a420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T00:35:12.781 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:12 smithi183 podman[71412]: 2024-09-11 00:35:12.426586997 +0000 UTC m=+0.015763177 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:35:12.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:12 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:12.548+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:12.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:12 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:12.548+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:12.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:12 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:12.548+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:12.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:12 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:12.548+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:12.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:12 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:12.548+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:12.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:12 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:12.548+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:13.281 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:12 smithi183 podman[71412]: 2024-09-11 00:35:12.784371583 +0000 UTC m=+0.373547763 container init 6e635d8921263505db1340361e7a1b5b07c89467029dd9c55825d86e47f6a420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:13.281 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:12 smithi183 podman[71412]: 2024-09-11 00:35:12.78753777 +0000 UTC m=+0.376713952 container start 6e635d8921263505db1340361e7a1b5b07c89467029dd9c55825d86e47f6a420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:13.281 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:12 smithi183 podman[71412]: 2024-09-11 00:35:12.876167744 +0000 UTC m=+0.465343921 container attach 6e635d8921263505db1340361e7a1b5b07c89467029dd9c55825d86e47f6a420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-11T00:35:13.281 INFO:journalctl@ceph.osd.8.smithi183.stdout:Sep 11 00:35:12 smithi183 podman[71412]: 2024-09-11 00:35:12.976917419 +0000 UTC m=+0.566093605 container died 6e635d8921263505db1340361e7a1b5b07c89467029dd9c55825d86e47f6a420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:35:13.622 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:13 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:13.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:13.623 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:13 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:13.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:13.623 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:13 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:13.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:13.623 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:13 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:13.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:13.623 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:13 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:13.294+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:13.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T00:35:13.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:13.279+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:13.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:13.279+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:13.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:13.279+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:13.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:13.279+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:13.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:13.279+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:13.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:13 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:13.279+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:13.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:13.724 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T00:35:13.724 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T00:35:13.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.9 2024-09-11T00:35:13.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:13 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:13.584+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:13.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:13 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:13.584+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:13.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:13 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:13.584+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:13.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:13 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:13.584+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:13.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:13 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:13.584+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:13.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:13 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:13.584+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:14.131 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:13 smithi103 systemd[1]: Stopping Ceph osd.9 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.128+0000 7f2389200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.128+0000 7f2389200640 -1 osd.9 96 *** Got signal Terminated *** 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.128+0000 7f2389200640 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.308+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.308+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.308+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.308+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:14.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.308+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:14.421 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:14 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:14.308+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:14.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:14 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:14.247+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:14.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:14 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:14.247+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:14.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:14 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:14.247+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:14.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:14 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:14.247+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:14.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:14 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:14.247+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:14.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:14 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:14.596+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:14.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:14 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:14.596+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:14.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:14 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:14.596+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:14.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:14 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:14.596+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:14.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:14 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:14.596+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:14.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:14 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:14.596+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:15.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:15 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:15.234+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:15.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:15 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:15.234+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:15.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:15 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:15.234+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:15.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:15 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:15.234+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:15.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:15 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:15.234+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:15.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:15 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:15.335+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:15.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:15 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:15.335+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:15.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:15 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:15.335+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:15.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:15 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:15.335+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:15.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:15 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:15.335+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:15.671 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:15 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:15.335+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:15.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:15.581+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:15.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:15.581+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:15.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:15.581+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:15.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:15.581+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:15.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:15.581+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:15.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:15.581+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:15.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:15 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:15.581+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:16.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:16.264+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:16.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:16.264+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:16.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:16.264+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:16.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:16.264+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:16.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:16.264+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:16.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:16 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:16.344+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:16.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:16 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:16.344+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:16.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:16 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:16.344+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:16.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:16 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:16.344+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:16.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:16 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:16.344+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:16.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:16 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:16.344+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:16.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:16 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:16.601+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:16.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:16 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:16.601+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:16.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:16 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:16.601+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:16.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:16 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:16.601+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:16.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:16 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:16.601+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:16.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:16 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:16.601+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:16.862 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:16 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:16.601+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:17.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:17 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:17.235+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:17.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:17 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:17.235+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:17.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:17 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:17.235+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:17.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:17 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:17.235+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:17.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:17 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:17.235+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:17.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:17 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:17.235+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:17.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:17 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:17.309+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:17.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:17 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:17.309+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:17.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:17 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:17.309+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:17.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:17 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:17.309+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:17.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:17 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:17.309+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:17.670 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:17 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:17.309+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:17.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:17.641+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:17.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:17.641+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:17.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:17.641+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:17.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:17.641+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:17.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:17.641+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:17.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:17.641+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:17.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:17 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:17.641+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:18.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:18 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:18.275+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:18.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:18 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:18.275+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:18.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:18 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:18.275+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:18.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:18 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:18.275+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:18.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:18 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:18.275+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:18.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:18 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:18.275+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:18.571 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:18 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:18.300+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:03.180545+0000 front 2024-09-11T00:34:03.180583+0000 (oldest deadline 2024-09-11T00:34:27.880682+0000) 2024-09-11T00:35:18.571 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:18 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:18.300+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:12.581669+0000 front 2024-09-11T00:34:12.581817+0000 (oldest deadline 2024-09-11T00:34:34.881465+0000) 2024-09-11T00:35:18.571 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:18 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:18.300+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:20.083683+0000 front 2024-09-11T00:34:20.083627+0000 (oldest deadline 2024-09-11T00:34:42.983537+0000) 2024-09-11T00:35:18.571 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:18 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:18.300+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.685472+0000 front 2024-09-11T00:34:28.685446+0000 (oldest deadline 2024-09-11T00:34:52.785532+0000) 2024-09-11T00:35:18.571 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:18 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:18.300+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:32.786293+0000 front 2024-09-11T00:34:32.786384+0000 (oldest deadline 2024-09-11T00:35:02.186995+0000) 2024-09-11T00:35:18.571 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:18 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9[66261]: 2024-09-11T00:35:18.300+0000 7f2384c00640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:44.988862+0000 front 2024-09-11T00:34:44.988879+0000 (oldest deadline 2024-09-11T00:35:09.088962+0000) 2024-09-11T00:35:18.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:18 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:18.594+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:18.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:18 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:18.594+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:18.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:18 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:18.594+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:18.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:18 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:18.594+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:18.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:18 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:18.594+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:18.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:18 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:18.594+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:18.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:18 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:18.594+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:19.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:19.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:19.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:19.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:19.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:19.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:19.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:19.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:19.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:19.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:19.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:19.316+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:19.909 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:19 smithi103 podman[117073]: 2024-09-11 00:35:19.749312826 +0000 UTC m=+5.735226295 container died f4f29ec5dfb83ddbf8cd71a190d7ec59799af443801769cbed2976e80b648f8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T00:35:19.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:19 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:19.609+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:19.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:19 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:19.609+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:19.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:19 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:19.609+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:19.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:19 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:19.609+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:19.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:19 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:19.609+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:19.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:19 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:19.609+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:19.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:19 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:19.609+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:20.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:20 smithi103 podman[117073]: 2024-09-11 00:35:20.167465599 +0000 UTC m=+6.153379069 container cleanup f4f29ec5dfb83ddbf8cd71a190d7ec59799af443801769cbed2976e80b648f8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T00:35:20.420 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:20 smithi103 bash[117073]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9 2024-09-11T00:35:20.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:20 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:20.271+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:20.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:20 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:20.271+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:20.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:20 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:20.271+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:20.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:20 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:20.271+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:20.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:20 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:20.271+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:20.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:20 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:20.271+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:20.920 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:20 smithi103 podman[117092]: 2024-09-11 00:35:20.455823163 +0000 UTC m=+0.692098612 container remove f4f29ec5dfb83ddbf8cd71a190d7ec59799af443801769cbed2976e80b648f8e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T00:35:20.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:20 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:20.639+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:20.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:20 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:20.639+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:20.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:20 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:20.639+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:20.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:20 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:20.639+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:20.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:20 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:20.639+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:20.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:20 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:20.639+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:20.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:20 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:20.639+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:21.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:21 smithi103 podman[117183]: 2024-09-11 00:35:21.011227346 +0000 UTC m=+0.027793555 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:35:21.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:21 smithi103 podman[117183]: 2024-09-11 00:35:21.194920593 +0000 UTC m=+0.211486786 container create 4ecfa34af4122746feb6b54c9dc96903cb91a86c72ef1d4a4b2757469910002d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T00:35:21.407 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:21 smithi103 podman[117183]: 2024-09-11 00:35:21.405980198 +0000 UTC m=+0.422546392 container init 4ecfa34af4122746feb6b54c9dc96903cb91a86c72ef1d4a4b2757469910002d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T00:35:21.666 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:21 smithi103 podman[117183]: 2024-09-11 00:35:21.413057564 +0000 UTC m=+0.429623758 container start 4ecfa34af4122746feb6b54c9dc96903cb91a86c72ef1d4a4b2757469910002d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:21.666 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:21 smithi103 podman[117183]: 2024-09-11 00:35:21.525390484 +0000 UTC m=+0.541956687 container attach 4ecfa34af4122746feb6b54c9dc96903cb91a86c72ef1d4a4b2757469910002d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:21.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:21.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:21.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:21.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:21.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:21.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:21.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:21.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:21.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:21.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:21.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:21 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:21.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:21.920 INFO:journalctl@ceph.osd.9.smithi103.stdout:Sep 11 00:35:21 smithi103 podman[117183]: 2024-09-11 00:35:21.665549757 +0000 UTC m=+0.682115971 container died 4ecfa34af4122746feb6b54c9dc96903cb91a86c72ef1d4a4b2757469910002d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:35:21.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:21 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:21.644+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:21.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:21 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:21.644+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:21.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:21 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:21.644+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:21.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:21 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:21.644+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:21.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:21 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:21.644+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:21.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:21 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:21.644+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:21.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:21 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:21.644+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:22.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T00:35:22.553 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:22.553 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T00:35:22.553 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T00:35:22.553 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.10 2024-09-11T00:35:22.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:22 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:22.322+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:22.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:22 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:22.322+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:22.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:22 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:22.322+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:22.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:22 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:22.322+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:22.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:22 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:22.322+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:22.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:22 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:22.322+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:22.893 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 systemd[1]: Stopping Ceph osd.10 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:22.893 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.658+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:22.893 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.658+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:22.893 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.658+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:22.893 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.658+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:22.893 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.658+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:22.893 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.658+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:22.894 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.658+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:22.894 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.890+0000 7f0557ae0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:22.894 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.890+0000 7f0557ae0640 -1 osd.10 96 *** Got signal Terminated *** 2024-09-11T00:35:22.894 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:22 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:22.890+0000 7f0557ae0640 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:35:23.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:23.280+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:23.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:23.280+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:23.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:23.280+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:23.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:23.280+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:23.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:23.280+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:23.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:23.280+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:23.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:23 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:23.651+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:23.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:23 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:23.651+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:23.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:23 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:23.651+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:23.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:23 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:23.651+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:23.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:23 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:23.651+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:23.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:23 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:23.651+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:23.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:23 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:23.651+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:24.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:24 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:24.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:24.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:24 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:24.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:24.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:24 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:24.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:24.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:24 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:24.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:24.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:24 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:24.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:24.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:24 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:24.270+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:24.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:24.623+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:24.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:24.623+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:24.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:24.623+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:24.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:24.623+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:24.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:24.623+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:24.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:24.623+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:24.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:24 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:24.623+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:25.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:25 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:25.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:25.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:25 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:25.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:25.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:25 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:25.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:25.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:25 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:25.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:25.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:25 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:25.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:25.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:25 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:25.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:25.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:25 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:25.266+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:25.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:25 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:25.646+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:25.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:25 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:25.646+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:25.937 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:25 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:25.646+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:25.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:25 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:25.646+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:25.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:25 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:25.646+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:25.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:25 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:25.646+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:25.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:25 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:25.646+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:26.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:26.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:26.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:26.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:26.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:26.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:26.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:26.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:26.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:26.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:26.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:26.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:26.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:26 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:26.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:26.891 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:26 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:26.619+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:26.891 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:26 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:26.619+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:26.891 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:26 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:26.619+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:26.892 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:26 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:26.619+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:26.892 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:26 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:26.619+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:26.892 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:26 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:26.619+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:26.892 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:26 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:26.619+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:27.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:27.288+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:27.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:27.288+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:27.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:27.288+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:27.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:27.288+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:27.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:27.288+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:27.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:27.288+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:27.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:27.288+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:27.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:27 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:27.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.733373+0000 front 2024-09-11T00:34:04.733410+0000 (oldest deadline 2024-09-11T00:34:29.433063+0000) 2024-09-11T00:35:27.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:27 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:27.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:13.434358+0000 front 2024-09-11T00:34:13.434417+0000 (oldest deadline 2024-09-11T00:34:38.134539+0000) 2024-09-11T00:35:27.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:27 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:27.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:18.135027+0000 front 2024-09-11T00:34:18.135169+0000 (oldest deadline 2024-09-11T00:34:44.035076+0000) 2024-09-11T00:35:27.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:27 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:27.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6814 osd.3 since back 2024-09-11T00:34:28.136220+0000 front 2024-09-11T00:34:28.136137+0000 (oldest deadline 2024-09-11T00:34:54.036207+0000) 2024-09-11T00:35:27.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:27 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:27.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:36.937095+0000 front 2024-09-11T00:34:36.937137+0000 (oldest deadline 2024-09-11T00:35:00.437168+0000) 2024-09-11T00:35:27.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:27 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:27.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:45.638667+0000 front 2024-09-11T00:34:45.638575+0000 (oldest deadline 2024-09-11T00:35:08.538580+0000) 2024-09-11T00:35:27.938 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:27 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10[60102]: 2024-09-11T00:35:27.590+0000 7f05540f5640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:54.439482+0000 front 2024-09-11T00:34:54.439589+0000 (oldest deadline 2024-09-11T00:35:15.539379+0000) 2024-09-11T00:35:28.559 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:28 smithi117 podman[70452]: 2024-09-11 00:35:28.281876688 +0000 UTC m=+5.529698480 container died 1caf242f3e93c10550eac117413ed9c4bddda0f64d41229bcaba36bd2d5db588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T00:35:28.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:28.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:28.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:28.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:28.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:28.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:28.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:28.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:28.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:28.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:28.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:28.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:28.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:28 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:28.293+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:28.892 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:28 smithi117 podman[70452]: 2024-09-11 00:35:28.648514915 +0000 UTC m=+5.896336709 container cleanup 1caf242f3e93c10550eac117413ed9c4bddda0f64d41229bcaba36bd2d5db588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:35:28.892 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:28 smithi117 bash[70452]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10 2024-09-11T00:35:29.187 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:28 smithi117 podman[70468]: 2024-09-11 00:35:28.891254063 +0000 UTC m=+0.605864336 container remove 1caf242f3e93c10550eac117413ed9c4bddda0f64d41229bcaba36bd2d5db588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T00:35:29.487 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:29 smithi117 podman[70556]: 2024-09-11 00:35:29.208825578 +0000 UTC m=+0.106541453 container create 05ebbcf0d40caf00f30d1afb0190523f98e42d4f68fcf404a9908304af4cb839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T00:35:29.487 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:29 smithi117 podman[70556]: 2024-09-11 00:35:29.117859539 +0000 UTC m=+0.015575368 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:35:29.487 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:29 smithi117 podman[70556]: 2024-09-11 00:35:29.410983024 +0000 UTC m=+0.308698854 container init 05ebbcf0d40caf00f30d1afb0190523f98e42d4f68fcf404a9908304af4cb839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:29.487 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:29 smithi117 podman[70556]: 2024-09-11 00:35:29.414099162 +0000 UTC m=+0.311814994 container start 05ebbcf0d40caf00f30d1afb0190523f98e42d4f68fcf404a9908304af4cb839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T00:35:29.487 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:29 smithi117 podman[70556]: 2024-09-11 00:35:29.485883714 +0000 UTC m=+0.383599559 container attach 05ebbcf0d40caf00f30d1afb0190523f98e42d4f68fcf404a9908304af4cb839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T00:35:29.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:29.339+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:29.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:29.339+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:29.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:29.339+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:29.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:29.339+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:29.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:29.339+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:29.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:29.339+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:29.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:29 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:29.339+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:29.890 INFO:journalctl@ceph.osd.10.smithi117.stdout:Sep 11 00:35:29 smithi117 podman[70556]: 2024-09-11 00:35:29.60438465 +0000 UTC m=+0.502100480 container died 05ebbcf0d40caf00f30d1afb0190523f98e42d4f68fcf404a9908304af4cb839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T00:35:30.159 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T00:35:30.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:30.198 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T00:35:30.198 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T00:35:30.198 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.11 2024-09-11T00:35:30.507 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 systemd[1]: Stopping Ceph osd.11 for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.301+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.301+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.301+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.301+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.301+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.301+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.301+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.505+0000 7f87c6361640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.505+0000 7f87c6361640 -1 osd.11 96 *** Got signal Terminated *** 2024-09-11T00:35:30.508 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:30 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:30.505+0000 7f87c6361640 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T00:35:31.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:31.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:31.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:31.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:31.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:31.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:31.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:31.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:31.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:31.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:31.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:31.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:31.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:31 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:31.306+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:32.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:32.272+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:32.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:32.272+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:32.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:32.272+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:32.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:32.272+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:32.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:32.272+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:32.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:32.272+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:32.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:32 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:32.272+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:33.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:33.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:33.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:33.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:33.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:33.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:33.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:33.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:33.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:33.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:33.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:33.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:33.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:33 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:33.318+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:34.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:34 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:34.365+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-11T00:35:08.397399+0000 front 2024-09-11T00:35:08.397420+0000 (oldest deadline 2024-09-11T00:35:33.697343+0000) 2024-09-11T00:35:35.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6806 osd.0 since back 2024-09-11T00:34:04.387670+0000 front 2024-09-11T00:34:04.387695+0000 (oldest deadline 2024-09-11T00:34:29.687853+0000) 2024-09-11T00:35:35.781 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6804 osd.1 since back 2024-09-11T00:34:09.688207+0000 front 2024-09-11T00:34:09.688187+0000 (oldest deadline 2024-09-11T00:34:35.588398+0000) 2024-09-11T00:35:35.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6804 osd.2 since back 2024-09-11T00:34:21.489408+0000 front 2024-09-11T00:34:21.489474+0000 (oldest deadline 2024-09-11T00:34:41.989538+0000) 2024-09-11T00:35:35.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6812 osd.4 since back 2024-09-11T00:34:37.692300+0000 front 2024-09-11T00:34:37.692464+0000 (oldest deadline 2024-09-11T00:35:03.492920+0000) 2024-09-11T00:35:35.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6812 osd.5 since back 2024-09-11T00:34:43.493301+0000 front 2024-09-11T00:34:43.493262+0000 (oldest deadline 2024-09-11T00:35:08.193399+0000) 2024-09-11T00:35:35.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.103:6822 osd.6 since back 2024-09-11T00:34:51.594775+0000 front 2024-09-11T00:34:51.594752+0000 (oldest deadline 2024-09-11T00:35:16.294670+0000) 2024-09-11T00:35:35.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.117:6820 osd.7 since back 2024-09-11T00:35:03.196577+0000 front 2024-09-11T00:35:03.196525+0000 (oldest deadline 2024-09-11T00:35:24.296499+0000) 2024-09-11T00:35:35.782 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T00:35:35.321+0000 7f87c2976640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.183:6820 osd.8 since back 2024-09-11T00:35:08.397399+0000 front 2024-09-11T00:35:08.397420+0000 (oldest deadline 2024-09-11T00:35:33.697343+0000) 2024-09-11T00:35:36.123 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:35 smithi183 podman[71525]: 2024-09-11 00:35:35.831156994 +0000 UTC m=+5.438490367 container died 3c6da5c8bdcab47021b444ec5f3cb8de9939f468cb9d4538736bf2e10922d7fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:35:36.123 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 podman[71525]: 2024-09-11 00:35:36.121990772 +0000 UTC m=+5.729324150 container cleanup 3c6da5c8bdcab47021b444ec5f3cb8de9939f468cb9d4538736bf2e10922d7fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0) 2024-09-11T00:35:36.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 bash[71525]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11 2024-09-11T00:35:36.531 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 podman[71541]: 2024-09-11 00:35:36.33137956 +0000 UTC m=+0.496810842 container remove 3c6da5c8bdcab47021b444ec5f3cb8de9939f468cb9d4538736bf2e10922d7fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:35:36.985 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 podman[71627]: 2024-09-11 00:35:36.562222559 +0000 UTC m=+0.016090653 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-11T00:35:36.985 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 podman[71627]: 2024-09-11 00:35:36.665505513 +0000 UTC m=+0.119373648 container create 5e7c52464974b96b35313557722f3956c861840290fb9fc8ee8a75dd61987e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-11T00:35:36.985 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 podman[71627]: 2024-09-11 00:35:36.895560008 +0000 UTC m=+0.349428101 container init 5e7c52464974b96b35313557722f3956c861840290fb9fc8ee8a75dd61987e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:36.986 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 podman[71627]: 2024-09-11 00:35:36.898971844 +0000 UTC m=+0.352839941 container start 5e7c52464974b96b35313557722f3956c861840290fb9fc8ee8a75dd61987e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:35:36.986 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:36 smithi183 podman[71627]: 2024-09-11 00:35:36.984242409 +0000 UTC m=+0.438110523 container attach 5e7c52464974b96b35313557722f3956c861840290fb9fc8ee8a75dd61987e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T00:35:37.281 INFO:journalctl@ceph.osd.11.smithi183.stdout:Sep 11 00:35:37 smithi183 podman[71627]: 2024-09-11 00:35:37.09002037 +0000 UTC m=+0.543888478 container died 5e7c52464974b96b35313557722f3956c861840290fb9fc8ee8a75dd61987e79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T00:35:37.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T00:35:37.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:37.720 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T00:35:37.720 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T00:35:37.720 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.a 2024-09-11T00:35:38.170 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:35:37 smithi103 systemd[1]: Stopping Ceph mds.a for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:38.670 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:35:38 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a[73267]: 2024-09-11T00:35:38.180+0000 7fbdd2a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:38.670 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:35:38 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a[73267]: 2024-09-11T00:35:38.180+0000 7fbdd2a00640 -1 mds.a *** got signal Terminated *** 2024-09-11T00:35:39.780 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:35:39 smithi103 podman[117284]: 2024-09-11 00:35:39.453486474 +0000 UTC m=+1.440370371 container died 6151c5100799f3162206b79dba8d2d72fca96e8bab599940319fb561e9c456e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T00:35:40.170 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:35:40 smithi103 podman[117284]: 2024-09-11 00:35:40.002565121 +0000 UTC m=+1.989449022 container cleanup 6151c5100799f3162206b79dba8d2d72fca96e8bab599940319fb561e9c456e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-11T00:35:40.170 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:35:40 smithi103 bash[117284]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a 2024-09-11T00:35:40.630 INFO:journalctl@ceph.mds.a.smithi103.stdout:Sep 11 00:35:40 smithi103 podman[117296]: 2024-09-11 00:35:40.304556129 +0000 UTC m=+0.845604171 container remove 6151c5100799f3162206b79dba8d2d72fca96e8bab599940319fb561e9c456e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-a, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:35:40.651 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T00:35:40.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:40.722 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T00:35:40.722 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T00:35:40.722 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.d 2024-09-11T00:35:40.920 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:35:40 smithi103 systemd[1]: Stopping Ceph mds.d for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:41.420 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:35:41 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d[72074]: 2024-09-11T00:35:41.101+0000 7f34e3e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:41.420 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:35:41 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d[72074]: 2024-09-11T00:35:41.101+0000 7f34e3e00640 -1 mds.d *** got signal Terminated *** 2024-09-11T00:35:45.420 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:35:45 smithi103 podman[117402]: 2024-09-11 00:35:45.119637578 +0000 UTC m=+4.132431421 container died da856ad467f40cb5a5af0befdee1cf76470fbb19d4331bcc1aebbec160e455a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:35:45.920 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:35:45 smithi103 podman[117402]: 2024-09-11 00:35:45.598363005 +0000 UTC m=+4.611156852 container cleanup da856ad467f40cb5a5af0befdee1cf76470fbb19d4331bcc1aebbec160e455a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T00:35:45.920 INFO:journalctl@ceph.mds.d.smithi103.stdout:Sep 11 00:35:45 smithi103 bash[117402]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-d 2024-09-11T00:35:46.136 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T00:35:46.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:46.208 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T00:35:46.208 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T00:35:46.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.g 2024-09-11T00:35:46.616 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:35:46 smithi103 systemd[1]: Stopping Ceph mds.g for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:46.919 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:35:46 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g[72869]: 2024-09-11T00:35:46.613+0000 7f22bc600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:46.920 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:35:46 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g[72869]: 2024-09-11T00:35:46.613+0000 7f22bc600640 -1 mds.g *** got signal Terminated *** 2024-09-11T00:35:50.888 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:35:50 smithi103 podman[117536]: 2024-09-11 00:35:50.626787707 +0000 UTC m=+4.139459232 container died c198b4477c4ed00c74c56059c78fe300f1756bd873e4f3c21e78649753e4fa3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-11T00:35:51.170 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:35:51 smithi103 podman[117536]: 2024-09-11 00:35:51.060329948 +0000 UTC m=+4.573001472 container cleanup c198b4477c4ed00c74c56059c78fe300f1756bd873e4f3c21e78649753e4fa3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T00:35:51.170 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:35:51 smithi103 bash[117536]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g 2024-09-11T00:35:51.577 INFO:journalctl@ceph.mds.g.smithi103.stdout:Sep 11 00:35:51 smithi103 podman[117548]: 2024-09-11 00:35:51.31341286 +0000 UTC m=+0.676238559 container remove c198b4477c4ed00c74c56059c78fe300f1756bd873e4f3c21e78649753e4fa3e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T00:35:51.585 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T00:35:51.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:51.656 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T00:35:51.656 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T00:35:51.656 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.j 2024-09-11T00:35:51.920 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:35:51 smithi103 systemd[1]: Stopping Ceph mds.j for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:52.420 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:35:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j[71706]: 2024-09-11T00:35:52.093+0000 7f779c600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:52.420 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:35:52 smithi103 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j[71706]: 2024-09-11T00:35:52.093+0000 7f779c600640 -1 mds.j *** got signal Terminated *** 2024-09-11T00:35:56.420 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:35:56 smithi103 podman[117654]: 2024-09-11 00:35:56.10524759 +0000 UTC m=+4.186466645 container died 59160ec8d9488713897aa367834ef0749ef8e1756db8009341d897982d461d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:35:56.836 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:35:56 smithi103 podman[117654]: 2024-09-11 00:35:56.608083507 +0000 UTC m=+4.689302560 container cleanup 59160ec8d9488713897aa367834ef0749ef8e1756db8009341d897982d461d32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-11T00:35:56.836 INFO:journalctl@ceph.mds.j.smithi103.stdout:Sep 11 00:35:56 smithi103 bash[117654]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-j 2024-09-11T00:35:57.001 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T00:35:57.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:35:57.071 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T00:35:57.072 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T00:35:57.072 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.b 2024-09-11T00:35:57.437 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:35:57 smithi117 systemd[1]: Stopping Ceph mds.b for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:35:57.694 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:35:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: 2024-09-11T00:35:57.453+0000 7fac223a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:35:57.694 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:35:57 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b[65293]: 2024-09-11T00:35:57.453+0000 7fac223a1640 -1 mds.b *** got signal Terminated *** 2024-09-11T00:36:01.763 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:36:01 smithi117 podman[70660]: 2024-09-11 00:36:01.464597469 +0000 UTC m=+4.199778350 container died 0c50953b2ef50d89f4b250a19e2bedae23bca2c1b5b514a64710a99f5d274be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T00:36:02.114 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:36:01 smithi117 podman[70660]: 2024-09-11 00:36:01.81953714 +0000 UTC m=+4.554718009 container cleanup 0c50953b2ef50d89f4b250a19e2bedae23bca2c1b5b514a64710a99f5d274be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T00:36:02.114 INFO:journalctl@ceph.mds.b.smithi117.stdout:Sep 11 00:36:01 smithi117 bash[70660]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-b 2024-09-11T00:36:02.242 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T00:36:02.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:02.277 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T00:36:02.278 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T00:36:02.278 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.e 2024-09-11T00:36:02.437 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:36:02 smithi117 systemd[1]: Stopping Ceph mds.e for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:36:02.937 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:36:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e[64680]: 2024-09-11T00:36:02.581+0000 7f5c1c8d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:36:02.938 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:36:02 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e[64680]: 2024-09-11T00:36:02.581+0000 7f5c1c8d9640 -1 mds.e *** got signal Terminated *** 2024-09-11T00:36:06.687 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:36:06 smithi117 podman[70797]: 2024-09-11 00:36:06.370521447 +0000 UTC m=+3.878186121 container died 6baff9b7466e6d17d9616fa99cb7b811fc4921696e2f73267525e5f8d834f471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:36:07.117 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:36:06 smithi117 podman[70797]: 2024-09-11 00:36:06.87662911 +0000 UTC m=+4.384293773 container cleanup 6baff9b7466e6d17d9616fa99cb7b811fc4921696e2f73267525e5f8d834f471 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-11T00:36:07.117 INFO:journalctl@ceph.mds.e.smithi117.stdout:Sep 11 00:36:06 smithi117 bash[70797]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-e 2024-09-11T00:36:07.247 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T00:36:07.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:07.281 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T00:36:07.281 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T00:36:07.281 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.h 2024-09-11T00:36:07.437 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:36:07 smithi117 systemd[1]: Stopping Ceph mds.h for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:36:07.937 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:36:07 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h[64980]: 2024-09-11T00:36:07.584+0000 7f7605ec7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:36:07.937 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:36:07 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h[64980]: 2024-09-11T00:36:07.584+0000 7f7605ec7640 -1 mds.h *** got signal Terminated *** 2024-09-11T00:36:10.687 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:36:10 smithi117 podman[70919]: 2024-09-11 00:36:10.376369573 +0000 UTC m=+2.881114870 container died 01b72f8cd469f41f376c1e3eac9216bcdcd808ac7b741323f9789e921f8ccfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:36:11.012 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:36:10 smithi117 podman[70919]: 2024-09-11 00:36:10.776431538 +0000 UTC m=+3.281176839 container cleanup 01b72f8cd469f41f376c1e3eac9216bcdcd808ac7b741323f9789e921f8ccfcf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:36:11.012 INFO:journalctl@ceph.mds.h.smithi117.stdout:Sep 11 00:36:10 smithi117 bash[70919]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-h 2024-09-11T00:36:11.153 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T00:36:11.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:11.186 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T00:36:11.186 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T00:36:11.186 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.k 2024-09-11T00:36:11.548 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:36:11 smithi117 systemd[1]: Stopping Ceph mds.k for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:36:11.937 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:36:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k[64366]: 2024-09-11T00:36:11.546+0000 7f1d37330640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:36:11.937 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:36:11 smithi117 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k[64366]: 2024-09-11T00:36:11.546+0000 7f1d37330640 -1 mds.k *** got signal Terminated *** 2024-09-11T00:36:15.915 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:36:15 smithi117 podman[71041]: 2024-09-11 00:36:15.561664704 +0000 UTC m=+4.162786600 container died f4c80cfd8ddd2bcc098143e404e8106866da52e3dd7e4b07a036e0bbf11764ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T00:36:16.187 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:36:16 smithi117 podman[71041]: 2024-09-11 00:36:16.021427824 +0000 UTC m=+4.622549708 container cleanup f4c80cfd8ddd2bcc098143e404e8106866da52e3dd7e4b07a036e0bbf11764ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T00:36:16.188 INFO:journalctl@ceph.mds.k.smithi117.stdout:Sep 11 00:36:16 smithi117 bash[71041]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-k 2024-09-11T00:36:16.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T00:36:16.430 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:16.430 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T00:36:16.430 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T00:36:16.430 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.c 2024-09-11T00:36:16.714 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:36:16 smithi183 systemd[1]: Stopping Ceph mds.c for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:36:16.967 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:36:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c[65748]: 2024-09-11T00:36:16.712+0000 7fcf000f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:36:16.968 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:36:16 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c[65748]: 2024-09-11T00:36:16.712+0000 7fcf000f6640 -1 mds.c *** got signal Terminated *** 2024-09-11T00:36:18.705 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:36:18 smithi183 podman[71732]: 2024-09-11 00:36:18.382196426 +0000 UTC m=+1.765115496 container died 5c800929afb3dadb1283d745d1613d867ea8c191c4decca3ec64a1804800934e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:36:19.030 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:36:18 smithi183 podman[71732]: 2024-09-11 00:36:18.795320233 +0000 UTC m=+2.178239290 container cleanup 5c800929afb3dadb1283d745d1613d867ea8c191c4decca3ec64a1804800934e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T00:36:19.031 INFO:journalctl@ceph.mds.c.smithi183.stdout:Sep 11 00:36:18 smithi183 bash[71732]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-c 2024-09-11T00:36:19.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T00:36:19.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:19.257 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T00:36:19.257 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T00:36:19.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.f 2024-09-11T00:36:19.600 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:36:19 smithi183 systemd[1]: Stopping Ceph mds.f for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:36:20.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:36:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: 2024-09-11T00:36:19.598+0000 7fb36a618640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:36:20.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:36:19 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f[65451]: 2024-09-11T00:36:19.598+0000 7fb36a618640 -1 mds.f *** got signal Terminated *** 2024-09-11T00:36:22.716 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:36:22 smithi183 podman[71866]: 2024-09-11 00:36:22.380118206 +0000 UTC m=+2.903933559 container died c8584de6ddb13ee604dfc1d755b561c5b612f7d855a5d924509d890cdaccb518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T00:36:22.716 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:36:22 smithi183 podman[71866]: 2024-09-11 00:36:22.715467407 +0000 UTC m=+3.239282759 container cleanup c8584de6ddb13ee604dfc1d755b561c5b612f7d855a5d924509d890cdaccb518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T00:36:23.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:36:22 smithi183 bash[71866]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f 2024-09-11T00:36:23.031 INFO:journalctl@ceph.mds.f.smithi183.stdout:Sep 11 00:36:22 smithi183 podman[71879]: 2024-09-11 00:36:22.924983219 +0000 UTC m=+0.537066571 container remove c8584de6ddb13ee604dfc1d755b561c5b612f7d855a5d924509d890cdaccb518 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T00:36:23.148 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T00:36:23.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:23.182 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T00:36:23.182 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T00:36:23.182 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.i 2024-09-11T00:36:23.521 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:36:23 smithi183 systemd[1]: Stopping Ceph mds.i for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:36:23.781 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:36:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i[66053]: 2024-09-11T00:36:23.518+0000 7feb26348640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:36:23.781 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:36:23 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i[66053]: 2024-09-11T00:36:23.518+0000 7feb26348640 -1 mds.i *** got signal Terminated *** 2024-09-11T00:36:26.645 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:36:26 smithi183 podman[71988]: 2024-09-11 00:36:26.37683572 +0000 UTC m=+2.983650090 container died 7c771aa545ac01e25ea61ce01be46f69b2d1148e2e7e63d2e442981270d5fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T00:36:26.996 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:36:26 smithi183 podman[71988]: 2024-09-11 00:36:26.735562886 +0000 UTC m=+3.342377256 container cleanup 7c771aa545ac01e25ea61ce01be46f69b2d1148e2e7e63d2e442981270d5fcd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=) 2024-09-11T00:36:26.996 INFO:journalctl@ceph.mds.i.smithi183.stdout:Sep 11 00:36:26 smithi183 bash[71988]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-i 2024-09-11T00:36:27.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T00:36:27.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:27.199 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T00:36:27.199 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T00:36:27.199 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.l 2024-09-11T00:36:27.524 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:36:27 smithi183 systemd[1]: Stopping Ceph mds.l for 40f10072-6fd2-11ef-bcea-c7b262605968... 2024-09-11T00:36:27.781 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:36:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l[65149]: 2024-09-11T00:36:27.521+0000 7fec6f2ec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T00:36:27.781 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:36:27 smithi183 ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l[65149]: 2024-09-11T00:36:27.521+0000 7fec6f2ec640 -1 mds.l *** got signal Terminated *** 2024-09-11T00:36:29.949 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:36:29 smithi183 podman[72108]: 2024-09-11 00:36:29.67849165 +0000 UTC m=+2.261269410 container died 82cb105a7b6ffa58e9ef6831a29378a28b7ef8535e951918e77ae800256bb27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-11T00:36:30.274 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:36:30 smithi183 podman[72108]: 2024-09-11 00:36:30.055667179 +0000 UTC m=+2.638444938 container cleanup 82cb105a7b6ffa58e9ef6831a29378a28b7ef8535e951918e77ae800256bb27d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T00:36:30.274 INFO:journalctl@ceph.mds.l.smithi183.stdout:Sep 11 00:36:30 smithi183 bash[72108]: ceph-40f10072-6fd2-11ef-bcea-c7b262605968-mds-l 2024-09-11T00:36:30.402 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-40f10072-6fd2-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T00:36:30.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T00:36:30.434 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T00:36:30.434 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T00:36:30.709 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:36:34.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T00:36:34.796 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:36:37.321 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T00:36:37.521 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:36:39.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:36:39.965 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:36:39.998 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T00:36:40.025 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T00:36:40.027 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi103/crash 2024-09-11T00:36:40.028 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash -- . 2024-09-11T00:36:40.103 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T00:36:40.103 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-09-11T00:36:40.106 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi117/crash 2024-09-11T00:36:40.107 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash -- . 2024-09-11T00:36:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T00:36:40.147 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-09-11T00:36:40.149 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi183/crash 2024-09-11T00:36:40.150 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash -- . 2024-09-11T00:36:40.189 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T00:36:40.189 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-09-11T00:36:40.189 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T00:36:40.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-11T00:36:40.277 INFO:tasks.cephadm:Compressing logs... 2024-09-11T00:36:40.278 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T00:36:40.282 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T00:36:40.285 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T00:36:40.310 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T00:36:40.310 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.11.log: 82.5% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T00:36:40.311 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T00:36:40.315 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log 2024-09-11T00:36:40.315 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.5.log: /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T00:36:40.315 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T00:36:40.315 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T00:36:40.315 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.l.log: 86.7% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T00:36:40.316 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-11T00:36:40.316 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T00:36:40.316 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T00:36:40.316 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T00:36:40.316 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T00:36:40.316 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T00:36:40.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T00:36:40.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T00:36:40.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T00:36:40.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T00:36:40.317 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T00:36:40.318 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T00:36:40.319 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.5% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T00:36:40.320 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T00:36:40.320 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log 2024-09-11T00:36:40.322 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.y.log: 95.0% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T00:36:40.322 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T00:36:40.323 INFO:teuthology.orchestra.run.smithi183.stderr: 95.1% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T00:36:40.323 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T00:36:40.323 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T00:36:40.324 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T00:36:40.324 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log: 86.7% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T00:36:40.325 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T00:36:40.327 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.h.log: 90.9% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T00:36:40.328 INFO:teuthology.orchestra.run.smithi117.stderr: 92.5% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T00:36:40.329 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.c.log: 92.6% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T00:36:40.330 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T00:36:40.330 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T00:36:40.331 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-11T00:36:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-09-11T00:36:40.332 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T00:36:40.333 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T00:36:40.334 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T00:36:40.334 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T00:36:40.335 INFO:teuthology.orchestra.run.smithi103.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-11T00:36:40.335 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T00:36:40.336 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T00:36:40.337 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T00:36:40.338 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T00:36:40.340 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T00:36:40.340 INFO:teuthology.orchestra.run.smithi103.stderr: 90.2% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T00:36:40.340 INFO:teuthology.orchestra.run.smithi103.stderr: 82.5% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T00:36:40.341 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T00:36:40.342 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.g.log: 2024-09-11T00:36:40.345 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T00:36:40.345 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T00:36:40.346 INFO:teuthology.orchestra.run.smithi103.stderr: 95.0% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T00:36:40.349 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T00:36:40.349 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T00:36:40.350 INFO:teuthology.orchestra.run.smithi103.stderr: 95.1% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T00:36:40.351 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T00:36:40.351 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.x.log: 86.7% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T00:36:40.351 INFO:teuthology.orchestra.run.smithi103.stderr: 91.0% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T00:36:40.352 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T00:36:40.355 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.j.log: /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.3.log: 95.1% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T00:36:40.356 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T00:36:40.361 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/cephadm.log: 95.0% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T00:36:40.364 INFO:teuthology.orchestra.run.smithi103.stderr: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T00:36:40.650 INFO:teuthology.orchestra.run.smithi103.stderr: 90.2% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T00:36:40.740 INFO:teuthology.orchestra.run.smithi183.stderr: 91.6% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T00:36:40.777 INFO:teuthology.orchestra.run.smithi117.stderr: 91.6% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T00:36:40.801 INFO:teuthology.orchestra.run.smithi117.stderr: 94.8% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T00:36:41.633 INFO:teuthology.orchestra.run.smithi117.stderr: 95.3% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T00:36:41.653 INFO:teuthology.orchestra.run.smithi103.stderr: 90.4% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T00:36:42.733 INFO:teuthology.orchestra.run.smithi183.stderr: 95.9% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T00:36:45.852 INFO:teuthology.orchestra.run.smithi183.stderr: 95.0% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T00:36:52.058 INFO:teuthology.orchestra.run.smithi103.stderr: 91.8% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T00:36:53.555 INFO:teuthology.orchestra.run.smithi183.stderr: 91.7% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T00:36:55.801 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T00:36:57.421 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T00:36:58.199 INFO:teuthology.orchestra.run.smithi117.stderr: 91.7% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T00:36:58.942 INFO:teuthology.orchestra.run.smithi103.stderr: 91.6% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T00:36:59.014 INFO:teuthology.orchestra.run.smithi117.stderr: 91.9% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T00:36:59.432 INFO:teuthology.orchestra.run.smithi183.stderr: 94.3% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T00:37:00.366 INFO:teuthology.orchestra.run.smithi117.stderr: 92.0% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T00:37:00.603 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T00:37:01.780 INFO:teuthology.orchestra.run.smithi103.stderr: 91.8% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T00:37:02.590 INFO:teuthology.orchestra.run.smithi103.stderr: 91.6% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T00:37:02.595 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-09-11T00:37:02.595 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m22.284s 2024-09-11T00:37:02.595 INFO:teuthology.orchestra.run.smithi103.stderr:user 1m9.994s 2024-09-11T00:37:02.595 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m5.758s 2024-09-11T00:37:07.981 INFO:teuthology.orchestra.run.smithi117.stderr: 93.5% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T00:37:07.983 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-11T00:37:07.983 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m27.684s 2024-09-11T00:37:07.983 INFO:teuthology.orchestra.run.smithi117.stderr:user 1m38.776s 2024-09-11T00:37:07.983 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m3.043s 2024-09-11T00:37:13.330 INFO:teuthology.orchestra.run.smithi183.stderr: 91.5% -- replaced with /var/log/ceph/40f10072-6fd2-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T00:37:13.332 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-09-11T00:37:13.332 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m33.033s 2024-09-11T00:37:13.332 INFO:teuthology.orchestra.run.smithi183.stderr:user 1m47.748s 2024-09-11T00:37:13.332 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m3.188s 2024-09-11T00:37:13.332 INFO:tasks.cephadm:Archiving logs... 2024-09-11T00:37:13.332 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi103/log 2024-09-11T00:37:13.333 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T00:37:25.578 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi117/log 2024-09-11T00:37:25.579 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T00:37:40.823 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi183/log 2024-09-11T00:37:40.824 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T00:37:56.353 INFO:tasks.cephadm:Removing cluster... 2024-09-11T00:37:56.354 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 --force 2024-09-11T00:37:56.634 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:37:57.362 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 --force 2024-09-11T00:37:57.567 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:37:57.951 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 40f10072-6fd2-11ef-bcea-c7b262605968 --force 2024-09-11T00:37:58.155 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 40f10072-6fd2-11ef-bcea-c7b262605968 2024-09-11T00:37:58.609 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T00:37:58.609 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T00:37:58.649 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T00:37:58.666 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T00:37:58.682 INFO:tasks.cephadm:Teardown complete 2024-09-11T00:37:58.682 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T00:37:58.695 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T00:37:58.695 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T00:37:58.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T00:37:58.709 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T00:37:58.852 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T00:37:58.863 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T00:37:58.863 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:37:58.868 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:37:58.870 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T00:37:58.883 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:37:58.885 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:37:58.888 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:37:58.888 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-11T00:37:58.888 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 13 -710us[ -712us] +/- 55ms 2024-09-11T00:37:58.888 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +983us[ +982us] +/- 53ms 2024-09-11T00:37:58.888 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -258us[ -258us] +/- 68ms 2024-09-11T00:37:58.888 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:37:58.889 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:37:58.889 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-11T00:37:58.889 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -671us[ -606us] +/- 55ms 2024-09-11T00:37:58.889 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +1109us[+1109us] +/- 53ms 2024-09-11T00:37:58.889 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 -256us[ -256us] +/- 68ms 2024-09-11T00:37:58.889 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:37:58.898 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-09-11T00:37:58.906 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T00:37:58.906 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-09-11T00:37:58.906 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 -712us[ -692us] +/- 55ms 2024-09-11T00:37:58.906 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 +992us[+1012us] +/- 53ms 2024-09-11T00:37:58.906 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -273us[ -253us] +/- 68ms 2024-09-11T00:37:58.907 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T00:37:58.907 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T00:37:58.919 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T00:37:58.920 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T00:37:58.933 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T00:37:59.079 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T00:37:59.151 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T00:37:59.217 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:37:59.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:37:59.279 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:37:59.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:37:59.309 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-11T00:37:59.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:37:59.337 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T00:37:59.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T00:37:59.361 INFO:teuthology.task.internal:Duration was 2042.281411 seconds 2024-09-11T00:37:59.362 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T00:37:59.374 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T00:37:59.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T00:37:59.378 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T00:37:59.381 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T00:37:59.419 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:37:59.420 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:37:59.462 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T00:37:59.736 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T00:37:59.736 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-09-11T00:37:59.736 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T00:37:59.785 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-09-11T00:37:59.786 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T00:37:59.807 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-11T00:37:59.807 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T00:37:59.827 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T00:37:59.828 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T00:37:59.832 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T00:37:59.849 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T00:37:59.894 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T00:37:59.894 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T00:37:59.898 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T00:37:59.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T00:38:00.592 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T00:38:00.604 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T00:38:00.604 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T00:38:00.660 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T00:38:00.686 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T00:38:00.714 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T00:38:00.724 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:38:00.729 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:38:00.731 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:38:00.752 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-09-11T00:38:00.779 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-11T00:38:00.795 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-09-11T00:38:00.824 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:38:00.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:38:00.858 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:38:00.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:38:00.873 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T00:38:00.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:38:00.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T00:38:00.899 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T00:38:00.901 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi103 2024-09-11T00:38:00.901 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T00:38:00.984 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi117 2024-09-11T00:38:00.984 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T00:38:01.021 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899823/remote/smithi183 2024-09-11T00:38:01.022 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T00:38:01.060 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T00:38:01.060 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T00:38:01.065 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T00:38:01.067 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T00:38:01.111 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T00:38:01.123 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T00:38:01.123 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T00:38:01.135 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T00:38:01.135 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T00:38:01.141 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T00:38:01.143 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T00:38:01.156 INFO:teuthology.orchestra.run.smithi117.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 00:38 /home/ubuntu/cephtest 2024-09-11T00:38:01.167 INFO:teuthology.orchestra.run.smithi183.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 00:38 /home/ubuntu/cephtest 2024-09-11T00:38:01.171 INFO:teuthology.orchestra.run.smithi103.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 00:38 /home/ubuntu/cephtest 2024-09-11T00:38:01.174 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T00:38:01.186 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T00:38:01.214 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 2042.281411409378 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-11T00:38:01.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T00:38:01.311 INFO:teuthology.run:pass